>. )) FOR NOVICE AND ADVANCED USERS 


LINUX JAILS 
IN PC-BSD 


INSTALLING AND CONFIGURING LINUX JAILS IN/PG-BSD= 

A SIMPLE DNS-DHCP SERVER FOR SMALL BUSINESS NETWORK WITH DNSMASO 
HARDENING FREEBSD WITH TRUSTEDBSD AND MANDATORY ACCESS CONTROLS 
EUROBSDCON AND MEETBSD CALIFORNIA: TWO CONTINENTS, ONE COMMUNITY 
FREEBSD ENTERPRISE SEARCH WITH APACHE SOLR 

POSTGRESQL: SCHEMAS 


VOL6 NO.T2 
ISSUE 12/2012(41) 
1898-9144 


800-820-BSDI 
http://www.iAsystems.com 
Enterprise Servers for Open Source 


UNIFIED. SCALABLE. FLEXIBLE. 


Across all industries the demands of data infrastructure have soared to new heights. 


As Capacity requirements continue to rise at an ever-increasing rate, performance must not be compromised. The hybrid 
architecture and advanced software capabilities of the TrueNAS appliance enable users to be more agile, effectively 
manage the explosion of unstructured data and deploy a centralized information storage infrastructure. Whether it’s 
backing virtual machines, business applications, or web services, there’s a TrueNAS appliance suited to the task. 


TrueNAS™ Storage Appliances: Harness The Cloud 


iXsystems’ TrueNAS Appliances offer scalable high-throughput, low latency storage 


All TrueNAS Storage Appliances feature the Intel® Xeon” Processors 5600 series, powering the fastest data transfer 
speeds and lowest latency possible. TrueNAS appliances come in three lines: Performance, Archiver, & High Availability. 
High-performance, high-capacity ioMemory modules from Fusion-io are available in the TrueNAS Enterprise, Ultimate, 


and Archiver Pro models. 


Py TruoNas 


Key Features: 


@ 


« One or Two Six-Core Intel® Xeon® Processors 
5600 series 

« Share Data over CIFS, NFS and iSCSI 

« Hybrid storage pool increases performance and 
decreases energy footprint 


* 128-bit ZFS file system with up to triple parity 
software RAID 


i 
Tron | [x] x} [x] | |. 
pedupication | | ||| « | || 
[igabitnics | Quad | Dual | Dual | Dual | Dual | sm | Qued | Dual 
Tocigabienics |_| dua [Qua | ova | || Ovar 


Call iXsystems toll free or visit our website today! 
1-855-GREP-4-IX | www.iXsystems.com 


Intel, the Intel loge, and Xeon Inside are trademarks or registered trademarks of Intel Corporation in the US. and other countries. 


CONTENTS ~ 


find | 
work eé 

In Decem 
popular series, 
Shirk (TrustedBSD), 
(PostgreSQL). All three 
BSD beta testers were asked to share 
for series. If there is any particular topic 
magazine’s bestseller — write us. 

On the last few pages are two overviews prest ants 
concerning BSD. Those who didn’t managed to atten them, 
have an opportunity to see, what they missed. . 

Since it’s Christmas time, together with our fellow editors 
from Hakin9 and PenTest Magazines, we prepared for you some 
presents! More you can find out in this issue from Hakin9 Magazine 
ad, so don’t miss it! 


Patrycja Przybytowicz 
Editor of BSD Magazine 
& BSD Team 


MAGAZINE 


BSD 


3 
= 


Editor in Chief: 
Ewa Dudzic 
ewa.dudzic@software.com.pl 


Supportive Editor 
Patrycja Przybytowicz 
patrycja.przybylowicz@software.com.pl 


Contributing: 
Rob Somerville, Kris Moore, Luca Ferrari, 
Antor i Francesco Gentile, Patrick Allen, ichael Dexter 


Top Betatesters & Proofre 
Barry Grumbine, Bjorn Michelsen, P q McMath, 
Imad Soltani, Luca Ferrari, Cleiton Alves, Eric Geissinger, 

i Kanth, Zander Hill, Ahmed Aneeth, Norman Golisz, 
Rob Cabrera, Will Clayton 


Speeral |\r as 
"RDS Tiae Daim 


Art Director: 
lreneusz Pogroszewski 


DTP: 
f Ireneusz Pogroszewski 
ireneusz.pogroszewski@software.com.pl 


Senior Consultant/Publisher: 
Pawel Marciniak pawel@software.com.pl 


Software Press Sp z 0.0. S s lool ing for partners from all 
over the world. If you are in erected’ in cooperation with us, 
please contact us via e-mail: editors@bsdmag.org. 


All trade marks presented in the magazine were used 
only for informative purposes. All rights to trade marks 
presented in the magazine are reserved by the companies 
which own them. 


Mathematical formulas created by Design Science 
MathType™. 


Let’s Talk 


OG The Sandbox 


By Rob Somerville 

8:45 Monday morning. | fill the espresso filter basket with 
a good measure of Italian coffee, flick the switch to espres- 
so, and 60 seconds later am rewarded with a demitasse 
of viscous caffeine, complete with the requisite creamy 
head. Coffee is an essential part of the I.T. toolkit, espe- 
cially when deadlines loom and the disconnect between 
customer, 3rd party supplier and the gap between expec- 
tations and reality becomes wider by the day... 


How To 
Installing and Configuring Linux Jails in 
08 PC-BSD” — 


By Patrick Allen 

Whether you prefer the CLI or a GUI, one thing most peo- 
ple can agree on, is that The Warden is a great tool for 
managing jails. The Warden has been available as an 
add-on in PC-BSD since version 8, and is available as 
a port in FreeBSD as well. It now comes built-in to ver- 
sion 9.1 of PC-BSD and TrueOS (a variant of PC-BSD 
included in the install DVD that consists of FreeBSD and 
enhanced command line versions of PC-BSD tools). 


1p FreeBSD Enterprise Search with Apache 
Solr (Part 4) 
By Rob Somerville 
So far, we have used Solr to access and index content 
found in web pages, XML files, databases and external 
websites. But as far as using Solr in the enterprise is con- 
cerned, how can we access disparate documents such as 
PDF and Microsoft Word files? This is where Apache Tika 
is invaluable — supporting over 14 different types of docu- 
ment formats. In the final part of our series on Apache Solr 
the author will look at Apache Tika and demonstrate how 
to import and index document content with Apache Solr. 


16 PostgreSQL: Schemas 
By Luca Ferrari 

This article provides an introduction to schemas, a fea- 
ture of PostgreSQL that allow Database Administrators 
(DBAs) to organize their database objects, mainly tables, 
into name spaces in order to either avoid naming conflicts 
and better structure the database itself. All the examples 
shown here have been tested on a PostgreSQL 9.1 clus- 
ter running on a FreeBSD 8.2-RELEASE machine; all the 
example source code is available in a GitHub repository. 
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28 Asimple DNS-DHCP Server for Small 
Business Network with dnsmasq 
By Antonio Franscesco Gentile 
From this article you will learn how to setup and manage 
a Small Business DNS/DHCP server. A real example of 
small LAN business network are the so called “SoHo” 
(single office/home office SOHO), namely a category of 
businesses that has 1 to 10 employees, but this is only 
the staring point. In fact, there are examples of deployable 
environment for Dnsmasq configurations used for more 
than 1000 hosts. 


security 


3G Hardening FreeBSD with TrustedBSD 
and Mandatory Access Controls (Part 4) 
By Michael Shirk 
Most system administrators understand the need to lock 
down permissions for files and applications. In addition to 
these configuration options on FreeBSD, there are fea- 
tures provided by TrustedBSD that add additional layers 
of specific security controls to fine tune the operating sys- 
tem for multilevel security. Since version 5.0 of FreeBSD, 
the TrustedBSD extensions have been included with the 
default install of the operating system. 


Overview 


49 EuroBSDcon and MeetBSD California: 
Two Continents, One Community 
By Michael Dexter 
This year’s EuroBSDcon and MeetBSD California took 
place just a few weeks apart in two very different locations 
but together demonstrated seamless solidarity on the part 
of the BSD community. MeetBSD in Sunnyvale, California 
was like a reunion for many speakers and attendees who 
had recently met in Warsaw, Poland for EuroBSDcon. 


AG PgDay.IT 2012 

By Luca Ferrari 
The sixth edition of the Italian PostgreSQL Day (PgDay) 
held at the Monash University Center in Prato, Tuscany, 
on November the 23th has been a success. The Italian 
community did respond very well to the event, and guests 
from all over the country came to discuss, acquire knowl- 
edge and share experience about this great database. 
Here is a great example of how passion can gather people 
together. Just follow their steps. 
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8:45 Monday morning. | fill the espresso filter basket with a good 
measure of Italian coffee, flick the switch to espresso, and 60 
seconds later am rewarded with a demitasse of viscous caffeine, 
complete with the requisite creamy head. Coffee is an essential 
part of the I.T. toolkit, especially when deadlines loom and the 
disconnect between customer, 3rd party supplier and the gap 
between expectations and reality becomes wider by the day. 


ods of patiently explaining this, while we can do the impossible, miracles 
take a little longer. 
| suppose the biggest curse of the technology sector is hype — unfortunately 
the “smoke and mirrors” brigade always seem to have the edge in persuad- 
ing the masses that technology is easy, close to infallible, and for X amount 
of money all your problems will be solved, and you will be a better person 
and a more efficient organisation as well. Sometimes corporate cultures re- 
ally excel in shooting themselves in the foot — hiring external consultants to 
“rubber stamp” strategic decisions that are not run past I.T. first, or worse 
still, delivered to them as a fait accompli with an impossible dead- 
line to match. 
Here starts the beginning of the disconnect. Technology 
is like a plant — it needs to be rooted in good soil, nur- 
tured and given the correct environment. Support 
structures need to be in place, weed killer em- 
ployed, and sometimes to get the best from the 
plant some serious pruning is required. 

Pests need to be controlled, symbiotic rela- 
tionships formed, and hopefully the ecosys- 
tem will be beneficial for the plant to flour- 
ish, thrive, and bear continuous fruit. In 
reality, sometimes the environment 

is harsh, short-cuts taken, essential 
maintenance ignored, critical invest- 
ment postponed and it is only a matter 
of time before the fire-fighting gets out 
of control and a major systems failure is 
experienced. Sometimes it is technology, but more 
often than not it is down to “expectations manage- 


start the week anticipating a diet of fire-fighting, cultural clashes and peri- 
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ment”. While the new system creeps past the line of “fit 
for purpose”, everyone knows it could have been so much 
better, more innovative, better engineered, future proofed. 
Designed by committee, any I.T. project is doomed to fail- 
ure unless everyone is on board and adheres to the cen- 
tral vision. In reality, this is rare unless there is a benevo- 
lent dictator to steer the process. This is where the hype 
does the most insidious damage — the commercial reali- 
ties of the vendor are to make a profit, to become an indis- 
pensable part of the customers’ ecosystem, while syphon- 
ing the last vestiges of innovation and creativity through 
restrictive licensing or security, intellectual property rights, 
and a “Yes we can do it but at a price” mentality. In this 
scenario, the vendor becomes the dictator, and the organ- 
isation is no longer in control. 
Once your most valuable resource (I.T.) is outside the 
doors of your organisation, you lose a crucial weapon 
in business — the ability to respond flexibly. 
Another layer has been added to the 
management structure, another 
form to fill in, another hand- 
holding session to explain 


Seiten a. Sate oa in words of one syllable 
> tak tage ans exactly what is required 
“tg — and why, another 

ra ~ specification to write 


(and to be ignored 
on the basis of be- 
ing commercially 
unrealistic for 

the vendor), 
another po- 
litical mine- 
field to ne- 
gotiate. 

In — short, 
rather 
than get- 


ting on with the job (designing, improving and developing 
systems), we are turning into project managers and ven- 
dor liaison officers, while watching efficiency slide and the 
corresponding shrinkage of morale and job satisfaction. 
“Can Do’ is replaced with “Not my responsibility” while the 
bitter spectre of how much company XYZ actually charged 
to modify and test 10 lines of code lies buried in some in- 
voice in the finance department. Roll up, roll up — all of 
these “benefits” can be yours too, if you outsource today. 

Fortunately, organisations are beginning to realise the 
folly of outsourcing. For all the rhetoric, good manage- 
ment boils down to one thing — control. Shrinking bud- 
gets are forcing companies to re-evaluate what value they 
get from suppliers, and the tide is turning towards Open 
Source and BSD unlike ever before. Smart businesses 
are building in-house teams, developing corporate loyalty, 
retaining staff and revolutionising their software platforms. 
Margaret Mead said “Never doubt that a small group of 
thoughtful, committed, citizens can change the world.” 
| say, give me half a dozen in-house BSD guru's, the free- 
dom to think “outside the box”, and we will transform your 
organisation. Austerity — sometimes — can work in your 
favour. 


ROB SOMERVILLE 

Rob Somerville has been passionate about technology since his 
early teens. A keen advocate of open systems since the mid eight- 
ies, he has worked in many corporate sectors including finance, 
automotive, airlines, government and media in a variety of roles 
from technical support, system administrator, developer, systems 
integrator and IT manager. He has moved on from CP/M and nixie 
tubes but keeps a soldering iron handy just in case. 
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Installing and Configuring 


Linux Jails in PC-BSD 


The Warden jail management tool has been redesigned for the 
upcoming release of PC-BSD 9.1. Many new features have 
been added, but one of the most exciting is the ability to create 


Linux jails. 


What you will learn... 

¢ Using Warden to create a Linux jail 
¢ Configuring nat for your jail 

¢ Installing Linux packages in the jail 


hether they prefer the CLI or a GUI, one thing 
VV most people can agree on is that The Warden 

is a great tool for managing jails. The Warden 
has been available as an add-on in PC-BSD since ver- 
sion 8, and is available as a port in FreeBSD as well. It 
now comes built-in to version 9.1 of PC-BSD and TrueOS 
(a variant of PC-BSD included in the install DVD that con- 
sists of FreeBSD and enhanced command line versions 
of PC-BSD tools). 


Background 

Jails are a very useful feature and a strong selling point 
of FreeBSD and derivative BSDs. OpenBSD, Linux and 
other U*ix operating systems typically use chroot to cre- 
ate 'safe' environments. In chroot environments, process- 
es are confined to a particular part of the host file system, 
and are not allowed to access files outside of it. Therefore, 
if a service running in a chroot is compromised, the host 
system should be safe from the attack. 

Jails take this a step further. In addition to the file sys- 
tem, jails virtualize other resources such as system users, 
running processes, the networking subsystem and more. 
Each jail even has its own root user. Jails do not provide 
a completely virtualized environment, as each jail sharing 
the host's kernel. 

Linux jails offer an interesting alternative for BSD us- 
ers who wish to create a virtualized environment. Some 
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What you should know... 


¢ Basic jail and *nix concepts 


users might wish to run services in a Linux environ- 
ment but don't have the resources, or the desire, to 
maintain a separate Linux server. While this was pos- 
sible in the past with a fair amount of effort, the new 
version of The Warden makes this easy, and in a very 
secure way. 

A thing to keep in mind about Linux jails however is 
that they are not actually utilizing a Linux kernel, so run- 
ning services that require specific Linux kernel func- 
tions are not possible. This also means that if you need 
to make kernel modifications in order to run a particu- 
lar program, these will need to be made to the host, 
BSD, kernel. 


Preparation 
Before we create our jail, there are a few things we need 
to take care of in order to allow it to access the internet 
and give us the ability to install packages. There are vari- 
ous ways to handle networking for jails, but for this exam- 
ple we will be using a loopback device, which we will call 
lo1. By creating this cloned interface we are giving the jail 
its own virtual network adapter which we can then config- 
ure separately from the actual physical adapter. For our 
jail in this example we will be using the address 10.0.0.1 
and will only be configuring IPv4. 

The first thing we need to do is create and configure the 
loopback device at the command line: 
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# ifconfig lol create 


# itcontig Lol 10.0.0.2 netmask 255.255.255.255 


To make this persistent, add the following to /etc/ 


re .cone. 


cloned: interfaces="Lol” 


ifconng. Jol="1net 10.0.0.2 netmask 295.255.255.255" 


Next, we will add rules to /etc/pf.con¢ to allow the jail to 
use nat by mapping it to out external interface (our de- 
fault ethernet adapter, emo). Assuming you are using the 
default /etc/pf.conf configuration file, the beginning of 
our file will now look like Figure 1. 

The lines we added are: 


* ext if="emo" — A macro for our external interface 
so that if we switch interfaces in the future, all refer- 
ences in the file to that interface will not need to be 
changed individually, 

¢* jail if="loi" — A macro for our loopback device 
that we just created, 


Terminal = reot#PCASoa1:— 


Configuration 
Jail Network Interface 
em0 (+ 
Jail Directory 
fusr/jails fam 
Temp Directory 


fusr/jails en 


Figure 2. Warden configuration 
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* jail ip="10.0.0.0/24" — A macro for our jail ip ad- 
dresses. This enables us to create more jails using 
the ip range of 10.0.0.0-10.0.0.24 which will then also 
use the nat that we are setting up, 

¢ natpassonsext if from sjail ip toany-> Sext _ 
if — Here we are configuring pf to nat all jail traffic. 


We also must assure that ip forwarding is enabled 
for IPv4: 


# sysctl -w net.inet.ip.forwarding=1 


To make this persistent, add the following to /etc/sysctl. 


Cont. 


net.inet.ip.forwarding=1 


New Jail Wizard 
¥ 


This wizard will walk you through creating a new jail. First, enter the new 
| IF address and hostname and click next to continue. 


IP Address 


| 10.0.0.2| 


Hostname 


linuxjail 


Figure 3./P and Hostname configuration 
ew Ja ard eles 
ee Please select the type of jail you want to create. 
Jail Type 
Traditional jail (Secure, best for server applications) 
Ports jail (insecure, allows running X applications) 
®) Linux jail (Run Linux within in a jail) 
<Back )( Next> )( Cancel 


Figure 4. Jail type selection 
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Lastly, we need to reload the pz rules: 
+ prctl —£ /etc/pf.cont 


Installing the jail 

To begin creating our jail, start The Warden. For this ex- 
ample we will be using the GUI, which can be started from 
the PC-BSD Control Panel or from the CLI using pc-su 
warden gui. The first time you start The Warden, it will ask 
you to set the configuration. If you are using ethernet, the 
Jail Network Interface should default to emo, and we can 
use the default Jail and Temp Directory (Figure 2). 

To add a new jail, click the green plus button. This will 
start the New Jail Wizard. The first screen asks for the IP 
address and hostname of our new jail. We will use the ad- 
dress we configured for our lo1 interface (Figure 3). 


New jail Wizard lclolm 


‘Y Next enter the root password for this jail. 


Root Password 
6606660 


Root Password (Confirm) 


_< Back) Cancel 
Figure 5. Entering the root password 
= New Jail Wizard [=)[5)[x) 
a 7 
| Please select the Linux install script to use for this jail. 
| As 
im Select Script eles!) 


| Look iin: 


[Bi usrfocalshare/wardervfinus: installs |y Oo 2 @ (=) 


Computer it ¥ nae 
2 ae a zr “i486 
PF patria 


i 
i 
i 
] 


Filename: | debian-6-squeeze 


* | X Cancel 


Files of type: |Linux install files (*) 


| 
— 


Figure 6. Selecting an install script 
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On the next screen, we are asked which type of jail we 
would like to create. Select Linux Jail (Figure 4). 

We are then asked to enter a root password for the jail 
(Figure 5). 

The next screen asks us to select a Linux install script 
for building the jail. As of PC-BSD 9.1 RC2, there are two 
install scripts included, debian-6-squeeze and gentoo- 
stage3-i486. At release more may be included, or you can 
easily build your own for other distros. For this example 
we will be using Debian (Figure 6). 

Last, we are asked if we'd like to start the jail at system 
bootup. Make your selection and click Finish. The Warden 


| The Warden =e) [mn] 
File jails 
Installed Jails 

ai + Hostname Status Updates 

| 

ti a Creating jail: 10.0.0.2  [-)feye] 

bk Configuring props... S 


Lb Canfiguring whiptail,.. 

by Canifigeuriiney iftepelousrt... 

I Configuring libreadine6.. 

I: Cantigurnng logrotate... 

I: Configuring ae _ 

I Configuring gpgy.. 

> 1 Canhigurineg gnu 59... + 
hk Configuring aebian- ~archive-keyring... 

i; Configuring apt... 

k Configuring hap 1... 


Workin 1; Genfiguring apt-utibs... 
—— | ht: Canhiguring apt 
info) | i; Configuring tasisel data... 


b Configuring tassel 
Jail § | |; Base system installed successfully 
: pad evkdb: warn rig. unkricwr mot shall 
Jail Success! Linux jail — at fuer pails 1.0.0.2 
i pole root password onc 1000.2 Enter new UNIM passwerd: Retype new UNIX 
sword: passwd: password updated successfully 
coess! 


N Close 


Addstional IPs: tedit) 
Listening on Ports: 123 


Figure 7. Jail creation is complete 


m7 The Warden 


Updates 


| Jail * Hostname Status 


- = 
Working on jail: 10.0.0.2 


Jail Infermation 
jad Tripe: Linrwuscst anil 
Sire on Disk: 261MB 
Start at boot: Disabled 

Network information 
Sctive Connections: 0 
Additional IPs; tedii) 
Listening on Ports: £0 123 


Figure 8. Our list of jails 
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then creates our jail. When it is finished, we can close the 
window (Figure 7). 

Our jail is now created but not started. Start it by either 
right-clicking on the jail and selecting Start This Jail or by 
clicking on the blue Play button or at the terminal by issu- 
ing the command warden start 10.0.0.2 (remember to 
use the name of the jail instead of the hostname). 

Once the jail is started, let's look at the Tools tab. Here, 
we can launch a terminal to your jail, or use the Export Jail 
feature to create an export that you can import as a new 
jail at a later point. 

On the Snapshots tab, if you are using ZFS for your 
host, you can create, restore or mount ZFS snapshots of 
your jail. You can also schedule daily or even hourly snap- 
shots, as well as select how many days worth of sched- 
uled snapshots to keep (Figure 9). 

You will notice that with a Linux jail, the packages tab is 
grayed out. Warden does not (at least at this time) provide 
the ability to install Linux packages from the GUI, so we 
will need to choose Launch Terminal from the Tools tab 
and do it ourselves. 

Different Linux distros come with various package man- 
agement tools. For instance, Gentoo uses emerge, Red 
Hat based distros use rpm and yum, and so on. The pack- 
age management tools provided with a Debian Linux jail 
are apt-get and dpkg. Some of the most popular Linux 


Info Toots | | Snapshots | Packages 
Snapshots 
No snapshots available. You may create one below. 


c jestore | » Mount 0) nmMeawni sb Add _ Remove 
Scheduled Snapshots 
: t Frequency 


Try beer 
1% Rot 


Figure 9. Snapshot tab 


jail: 10.0.0.7 


jrootel inuclad]i/# apt-get update 


thtted-util 
The thi las MEM packages will be installed: 


JO upgraded. 1 newly inétal led, ee ee ee Se ee 
Need to get 0 BBS kB of arch 

\Airter thic operation, 172 hE ey wd ttonal disk epace will be ued, 
Selecting previously deselected 
(Reading 9140 Files and at directories currently installed.) 
Urpack ing thtted (Fron --fthttpd_2. 25b-11_kfreebed-i385..deb) . 


eistine 10. Using apt-get in a Linux ‘ai 
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distros are based on Debian, so there is much documen- 
tation available on using these tools. As an example we 
will install thttod, the tiny/turbo/throttling HTTP server, us- 
ing apt-get. 

At the command line in your jail terminal, we will first run 
apt-get update, which makes sure our package source 
lists are up to date. Next, to do the install we enter apt-get 
install thttpd. If we are installing a package with depen- 
dencies, apt-get will ask us if we'd like to install those de- 
pendencies before installing the package. You now have 
a Linux jail with the thttpd server installed. 


Conclusion 

Jails are a great tool for system administrators, giving them 
the portability of being able to easily copy or move the 
whole environment, and the security and stability of being 
isolated from the host system. They are a smart alterna- 
tive to traditional virtualization since they do not have the 
overhead of hardware emulation, providing a lightweight 
environment when performance is a priority. The addition 
of Linux adds to the usefulness and flexibility of jails, giv- 
ing administrators more options than ever to set up just 
the type of environment they need. The information and 
tools explained in this article should give a user the abil- 
ity to hit the ground running with Linux jails when PC-BSD 
9.1, and the new version of Warden, are available. To read 
more about the new features in the upcoming version, vis- 
it the preview version of the new Warden documentation 
at http://wiki.pcbsd.org/index.php/Warden. 


PATRICK ALLEN 

Patrick Allen is a developer and DBA from Colorado Springs, Col- 
orado. He uses AIX and Linux at work, and *BSD for pleasure. 
However, he still misses his first true love, his Commodore 64. 
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FreeBSD Enterprise 


Search with Apache Solr (Part 4) 


In the final part of our series on Apache Solr we will look at Apache 
Tika and demonstrate how to import and index document content 


with Apache Solr. 


What you will learn... 
¢ How to set up Apache Tika and integrate it with Solr 


tent found in web pages, XML files, databases and 

external websites. But as far as using Solr in the 
enterprise is concerned, how can we access disparate 
documents such as PDF and Microsoft Word files? This 
is where Apache Tika is invaluable — supporting over 14 
different types of document formats (Table 1 — Tika sup- 
ported document formats). 

Processing the files takes place in two stages. In stage 
one, Tika reads the file then parses and extracts the rel- 
evant meta-data. In the second stage the extracted data 
is posted to Solr. Searching the file comprises two stages, 
the query is sent to Solr by the user and Solr returns the 
link to the document. The user can then view or download 
the document via their browser (Figure 1 — Parsing, post- 
ing and querying). In the case of documents (PDF, DOC 
etc.) both the content and the meta-data is extracted. In 
the case of media files, archives etc, only the meta-data 
is extracted, for example exif data in the case of images, 
and MP3 tags in the case of MP3 files. 

Physical access to the documents can be achieved in a 
number of ways. In the case of a content management 
system, the file or attachment is uploaded via the CMS in- 
terface, and then processed and stored on the web-serv- 
er. Accessing files stored en masse on a file-server could 
be accomplished by using fusefs-ntfs and Apache. Irre- 
spective of method used, Tika must be able to read, parse 
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What you should know... 
¢« BSD administration skills, FreeBSD Apache Solr Parts 1, 2 and 
3 articles 


and post to Solr, and conversely Solr must be able to point 
back to the file either via a physical share (on an intranet 
for example) or via an HTTP link to the file. 


Table 1. Tika supported document formats 


Content and metadata extraction 


XML and derived formats XHTML, OOXML and ODF 


OpenDocument Format 


Electronic Publication Format EPUB 


Text formats TXT, CSV 


Metadata 


Compression and packaging bzip2, tar and zip 
formats 


Image formats JPG,PNG.GIF 


Java class files and archives JAR 
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Required Files 

If you have been following the series from the beginning, 
tika-core-1.1.jar and tika-parsers-1.1.jar should already 
be in the collectionx/1lib directory, with tika-app-1.2.jar 
in the tmp/solr directory. If not, Tika will have to be manu- 
ally downloaded and compiled using Maven. See (Table 
2/3). You will also need some sample files to import — in 
this example | will use the previous Solr 3 article in vari- 
ous file formats. 


Step 1. Configure Tika 

Log in to your test Solr server, stop your running Solr in- 
stance, and then create a new collection with the exten- 
sive schema from collection: 


su 
/usr/local/etc/rce.d/tomcat7 stop 


cd /home/solr 


S$ S$ SF SF 


cp -R collection3 collection4 


Creating the new collection 
Edit solr.xml to reflect the new collection by adding the fol- 
lowing lines to the <cores> section (Listing 1). 


# vi solr.xml 
Editing solr.xml (Listing 1). Change the cores line to read 


(Listing 2). Remove the line (Listing 3), and replace it 
with (Listing 4). Flush the index data: 


# rm collection4/data/index/* 


# rm collection4/data/tlog/* 


stage 1 


Tika 


Extract and 
pare oonbent 


Stage 2 


p= .. 


Stage 3 


Poet to Sole — Content = 6.9. Ipaum Lorem 


Result 


Figure 1. Parsing, posting and querying 
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Flushing the indexes 
Ensure that file and group rights are correct: 


# chown -R www:www collection4 


# ls -alh collection4/lib/tika* 


Updating the user rights and checking Tika JAR 
You should see 2 Tika JAR files present (Figure 2). 
Do not copy the tika-app jar in /tmp/solr to your lib direc- 
tory — this will cause the indexing to fail. 

We could use Tomcat to serve the files, but Apache 
is better for this. Install and configure it to point to the 
/exampledocs directory, and then restart Tomcat: 


# pkg add -r apache22 

# echo 'apache22 enable="YES"' >> /etc/rc.conf 

# rm -fr /usr/local/www/apache22/data/ 

# In -s /home/solr/exampledocs /usr/local/www/apache22/ 


data 


Listing 1. XML 


<core schema=”"schema.xml” 
instanceDir="/home/solr/collection4/” 
name="collection4” 
config="solrconfig. xml” 


dataDir="/home/solr/collection4/data” 


ic 

Listing 2. XML 

<cores adminPath="/admin/cores” zkClientTimeout="${zk 
ClientTimeout:15000}” > 

Listing 3. XML 


<core instanceDir="collection1/” name="collection1” /> 


Listing 4. XML 


<core schema="schema. xml” 
instanceDir=”"/home/solr/collection1/” 
name="collection1l” 
config="solrconfig. xml” 
dataDir=”"/home/solr/collection1/data” 

le 


lectionéd/Lib/tika- 
3 collectiond/Lib/tika-parsers-1.1.jar 


oLr shome/solr]# 


Figure 2. Tika JAR files 
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# /usr/local/etc/rc.d/apache22 restart 
¢ JusY/ local/étc/ re.d/tomcat/ start 


Installing Apache 

Now copy or create the test files into /nome/solr/ 
exampledocs on your server using FileZilla, or Midnight 
Commander etc. | used the convention solr.pdf, solr.doc, 
solr.txt etc. for this how-to. If you point your browser to 
http://yourserveripaddress you should see a directory list- 
ing similar to (Figure 3). Perform a quick check to make 
sure you can download / open the example files. Also, 
check that collection4 has come up. 


Step 2. Manually Test Tika and Create the 
Schema 

We now want to ensure that Tika can extract the meta- 
data from the files. You should see the output similar to 
(Figure 4). 


(>¢ 
Index of / 


[) 192.168.0.127 


FreeBSD Solr - Part 3.doc 
FreeBSD Solr - Part 3,docx 


GENERATED.xml 
gb18030-example.xml 
hd, xml 


ipod_other.xml 
ipod _video.xml 
mem. xml 
money.xml 
monitor. xml 
‘har? 


* ¢ #¢@ #@ #© # @#@ &#@ & £# &#®@h—hUhFH Ff &ehUhhmhUMhOhUCUchOMhCUc OhClCUcrrhUlUcrhOrLhUrhTrhUcr}mhUch}DMh hUch}VL[hUDh hm 


Figure 4. Tika Metadata example 
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# java -jar /tmp/solr/tika-app-1.2.jar -m \ 
/home/solr/exampledocs/solr.pdf 


Viewing metadata 

Repeat with the different filetypes to ishow how Tika auto- 
matically extracts the different types of meta-data from the 
files. To see how Tika extracts the content from the PDF 
run the following command: 


# java -jar /tmp/solr/tika-app-1.2.jar -t \ 
/home/solr/exampledocs/solr.pdf 


Viewing content 
Stop Tomcat and edit solrconfig.xml to reflect (Listing 5) 


# /usr/local/etc/rc.d/tomcat7 stop 


# vi collection4/conf/solrconfig.xml 


Editing solrconfig.xml (Listing 5). 
Add the following field to schema.xml under fields (List- 
ing 6). 


Step 3. Restart tomcat and Load the Files 
# /usr/local/etc/rc.d/tomcat7 stop 
Restarting Tomcat 


Using curl we will perform Stages 1 and 2 from (Figure 1) 
extracting both the content and the metadata from the files: 


Listing 5. XML 


<requestHandler name="/update/extract" 
class="solr.extraction.ExtractingRequestHandler" > 
<lst name="defaults"> 
<str name="fmap.a">links</str> 
<str name="fmap.div">ignored </str> 
<str name="fmap.content">text</str> 
<str name="lowernames">true</str> 


<str name="uprefix">attr_</str> 


<str name="captureAttr">true</str> 
<7 ket 


</requestHandler> 


Listing 6. XML 


<dy tom chelc name—attry*™ 
type="text general" 
indexed="true" 
stored="true" 


multiValued="true"/> 
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Figure 5. Solr results with metadata 


# curl "http://localhost:8080/solr/collection4/update/ 
extract? \ literal.id=l&commit=true" -F "myfile=@/home/ 


solr/exampledocs/solr.pdf" 


Extracting and parsing 
You should see the following: 


# <?xml version="1.0" encoding="UTF-8"?><response> 
<lst name="responseHeader"><int name="status">0 


</int><int name="QTime">750</int></lst></response> 


Solr response 

Repeat for each document you want Solr to process, in- 
crementing 1iteral.id by one each time and replacing 
Solt spdt with solr.rtf£ and solr.doc etc. 


Next Steps 

To make this production ready, you need to accurately de- 
fine the handlers used for each core and the schema for 
the files we want to search. A shell script could easily be 
used to load files en masse, it would just be a matter of 
writing some straightforward glue code in Perl or PHP as 
a basic front-end to pass the search query to Solr and 
return the metadata and a sample of the body content, 
along with the HTML link to the file. 


Conclusion 

In the past four articles we have demonstrated how pow- 
erful and flexible Solr, Tika and Nutch can be when used 
either side of the corporate firewall. When coupled with a 
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Table 2. Required JAR files 


File-name Components 


tika-parsers-1.1.jar Tika parsers 


Table 3. Further reading 


Tika download page 


http://tika.apache.org/download.html 


well designed schema, it is a robust and scalable solution 
to the ever expanding challenge of integrating systems. 
While designed primarily as a search framework, applica- 
tions from corporate website search through to document 
management are possible. With a little imagination (and if 
necessary some additional coding), Solr can truly be the 
central search facility for any organisation. 
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PostgreSQL: 


Schemas 


This article provides an introduction to schemas, a feature of 
PostgreSQL that allow Database Administrators (DBAs) to organize 
their database objects, mainly tables, into name spaces in order to 
either avoid naming conflicts and better structure the database itself. 
All the examples shown here have been tested on a PostgreSQL 9.1 
cluster running on a FreeBSD 8.2-RELEASE machine; all the example 
source code is available in a GitHub repository. 


What you will learn... 
« What schemas are and how to take advantage of them 
« How to organize your database objects into schemas 


feature of PostgreSQL that allow Database Admin- 

istrators (DBAs) to organize their database objects, 
mainly tables, into name spaces in order to either avoid 
naming conflicts and better structure the database itself. 
All the examples shown here have been tested on a Post- 
greSQL 9.1 cluster running on a FreeBSD 8.2-RELEASE 
machine; all the example source code is available in a 
GitHub repository. 


y his article provides an introduction to schemas, a 


What is a Schema? 

As detailed in previous articles in this series, a PostgreSQL 
instance can manage a cluster of databases, all logical- 
ly separated from each other. Within each database, the 
objects (tables, indexes, stored procedures, views, and 
So on) can be further split into named sets called sche- 
mas. Generally speaking, a schema can be thought as a 
“namespace” to contain database objects (mainly tables). 
Advantages of using schemas are mainly the followings: 


¢ provide a clear structure of the database, keeping ob- 
jects that are not strictly related to each other sepa- 
rated. AS an example, a configuration table should 
not be in the same place as an accounting table, 
since the two tables serve different purposes; 

¢ provides a level of granularity allowing a DBA to sepa- 
rately backup and restore a whole set of objects with- 
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What you should know... 

¢ Basic shell commands 

¢ Basic PostgreSQL concepts 

« Database object creation (creating tables, indexes, and so on) 


in a schema without having to touch the others, and to 
quickly set up permission grants on each set of objects 
addressing the whole schema they belong to; 

¢ avoid naming conflicts, allowing objects within differ- 
ent schemas to have the same name. 


In PostgreSQL each database object has to belong to a 
schema, therefore if the DBA or the developer does not 
explicitly specify any schema, PostgreSQL adopts a de- 
fault schema named public. A database object is there- 
fore fully qualified by its simple name and the name of 
the schema, with the schema coming first and with a dot 
"' aS separation, as follows: 


schemaName.objectSimpleName 


In a few cases the schema name can be omitted, and 
this is usually the case for the public schema (more on 
this later). In order to see the public schema in action 
consider the simple definition of the magazine table used 
in the previous articles’ examples and shown in Listing 
1. From a psql(1) terminal it is possible to see the def- 
inition of the table with the instrospection command \a 
magazine, aS shown in Listing 2: please note that the sys- 
tem reports the table with the prefix public, and there- 
fore the table is named public.magazine and not sim- 
ply magazine as in the creation instruction of Listing 1. 
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Listing 1.A magazine table definition 


DROP TABLE magazine CASCADE; 

CREATE TABLE IF NOT EXISTS magazine(pk serial NOT NULL, 
Tel Ge ete, 

month int, 

issuedon date, 

title text, 

PRIMARY KEY (pk), 

UNIQUE (id) 

); 


TRUNCATE TABLE magazine; 
INSERT INTO magazine (pk, id, month, issuedon, title) 
VALUES: 20d 01" | 1; OZ =Ol-Ol"sdate:; ‘FreeBSD: Get Up To Date’); 


INSERT INTO magazine (pk, id, month, issuedon, title) 
VALUES (2-201) 2 ‘2012-04-01’::date , ‘Rolling Your Own Kernel’); 


INSERT INTO magazine (pk, id, month, issuedon, title) 
VALVUES(3, 7 20 ab, ‘2011-01-01’::date, ‘Speed Daemons’); 


Listing 2. The full name of the table 


bsdmagdb=# \d magazine 
Table “public.magazine” 


Column | Type | Modifiers 
aes ee ee ee ee 
pk | integer | not null default nextval (‘magazine pk seq’::regclass) 
aC | | text | 
month | integer | 


issuedon | date | 

title | text | 

Indexes: 
“magazine pkey” PRIMARY KEY, btree (pk) 
“magazine id key” UNIQUE CONSTRAINT, btree (id) 


bsdmagdb=# SELECT * FROM magazine; 


eae | id | month | issuedon | title 

----}--------- +------- +------------ $+------------------------- 
dle | TO Chae 1 | 2012-01-01 | FreeBSD: Get Up To Date 
27 Olle ake || 12 | 2012-04-01 | Rolling Your Own Kernel 
ial ees Cale lela 11 | 2011-01-01 | Speed Daemons 

bsdmagdb=# SELECT * FROM public.magazine; 

pk | aC | month | issuedon | title 

~---}--------- +------- +------------ $------------------------- 
ee Oe lee 1 | 2012-01-01 | FreeBSD: Get Up To Date 
Ze OR | i | 2002-04-01 enolling Your “Own Kernel 
ol, - 20 ders P20 Oo | Speeenacnionis 
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Listing 3. Creating the three magazine tables in separated schemas 


CREATE 
CREATE 


pentestmag | bsdmag 

SCHEMA bsdmag; 

TABLE IF NOT EXISTS bsdmag.magazine( /* as in 
ites tate; 10 327) ))e- 


public | pgsql 


bsdmagdb=# \d 


CREATE SCHEMA pentestmag; Schema | Name 
CREATE TABLE IF NOT EXISTS pentestmag.magazine( /* as in  -------- fooaago soo ace tao fesaaeo ose 
lbs; le tecop ile 75) bsdmag | magazine 
CREATE SCHEMA linuxmag; bscmagy | Pmagazine pkyseq 
CREATE TABLE IF NOT EXISTS linuxmag.magazine( /* as in public | articles 
isting lp) publicw | asereles pikyced s| 
public | listini | 
Listing 4. Inspecting available tables public | listini pk seq | 
bsdmagdb=# \dn public | readers | 
ibs OL es chemas jeloibke! || weakens Jol Seo) | 
Name | Owner public | test | 
a a leas eats ae ae publie | test. ekysed | 
bsdmag | bsdmag 
linuxmag | bsdmag 


Listing 5. Ensuring all the tables are in the proper schema 
bsdmagdb=# \d public.magazine 


Table “public.magazine” 


Column | Type | Modifiers 
eet ener Spe em ee ge lite en Ne eae es Pt Srey ie Re cs Oe eet Meee ee epee ee ett Ree Ne See 
pk | integer | not null default nextval(‘public.magazine pk seq’::regclass) 
id | text | 
month | integer | 


issuedon | date | 


title | text | 


bsdmagdb=# \d pentestmag.magazine 


Table “pentestmag.magazine” 


Column | Type | Modifiers 
See operas I aa a cee OR a ea ON ee, SR Oe eR Ce 
pk | integer | not null default nextval(‘pentestmag.magazine pk seq’::regclass) 
id | text | 
month | integer | 


issuedon | date | 


title | text | 


bsdmagdb=# \d linuxmag.magazine 


Table “linuxmag.magazine” 


Column | Type | Modifiers 
ee Ee ae ee ee a ee eee 
pk | integer | not null default nextval(‘linuxmag.magazine pk seq’::regclass) 
10 | | text | 
month | integer | 


issuedon | date | 


title | text | 


hist of relations 


sequence 
table 
sequence 
table 
sequence 
table 
sequence 
table 


sequence 
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Every SQL command understand the usage of a sche- 
ma qualifier, and as shown in the bottom half of Listing 2, 
accessing the magazine table is possible either via the 
simple name magazine or the fully qualified name pub- 
lic. magazine. 


Using Schemas 

To better understand the advantages of using schemas, 
an ad-hoc example will be shown. Consider a database 
that will contain information about different published 
magazines, such as BSD Magazine, Linux Magazine, 
PenTest Magazine, and so on. All the magazines have a 
set of shared data, for instance the list of readers and au- 
thors, and a set of private data, such as each magazine 
title, issue, and so on. Instead of creating a different da- 
tabase for each set of data related to each magazine, the 
database will be only one, but each magazine will store 
its private data into a magazine-like table. This isn't the 
best real-world design, but it does suffice to explain how 
to solve some problems using schemas. Imagine also that 


PostgreSQL: Schemas 


there is a constraint to use the same table name for each 
magazine's private data, that is each magazine will store 
its data in a magazine called table within the same da- 
tabase. This means that in the database there will be at 
least three magazine tables, one for the BSD Magazine, 
one for the Linux Magazine, and one for the PenTest Mag- 
azine. The problems that arise from such a situation are (i) 
nameclashing, (ii) different permission handling, (iii) differ- 
ent backup strategies, while advantages are (i) the data- 
base is self contained, (ii) the structure of each set of data 
is exactly the same and (ili) each set can be profiled in a 
different way. 


Using Schemas to Solve Name Conflicts 

The first problem, name clashing, can be easily solved 
using schemas: it suffices to create a single schema 
for each magazine and to store the magazine table into 
this schema. A schema can be created with the create 
SCHEMA COmmand, which requires a name for the sche- 
ma (that of course has to be unique within the database). 


Listing 6. Viewing the current search_path and modifying it 


bsdmagdb=# SHOW search path; 

Sscanch earth 

“Suser” ,public 
bsdmagdb=# SET search path TO linuxmag, public; 
bsdmagdb=# SHOW search path; 

Sscanch) pat 

linuxmag, public 

bsdmagdb=# \d 
hist o£ relavions 

Schema _ | Name | | Owner 
~--------- $-----------------4----------4-------- 
linuxmag | magazine | table | bsdmag 
linuxmag | magazine pk seq | sequence | bsdmag 
public | articles | table | bsdmag 
bsdmagdo—7 sel Scarcm para lO™ sce  vinusmag, penis uma 

Gg, bsdmag, public; 
bsdmagdb=# SHOW search path; 
Sscarch path 


“Suser”, linuxmag, pentestmag, bsdmag, public 


(1 row) 


bsdmagdb=# \d 


vst Of relations 


bsdmag | magazine | table | bsdmag 


bsdmag | magazine pk seq | sequence | bsdmag 
| table 


public | articles | bsdmag 


Listing 7. [Inspecting which magazine table is available at any time 
depending on the search schema 


SELECT n.nspname, -- schema name 
c.relname, -- relation name 
e ealicl —— relarion coud 
jie) elcellog- joe) eso Ils aks Valse || Weealcl == is icles 


schema in the search path? 
FROM pg class c 
LEFT JOIN pg catalog.pg namespace n 
ON n.oid = c.relnamespace 
WHERE c.relname = ‘magazine’ 


ORDER BY n.nspname; 


nspname [evelname 9 |) Sond | jet wallolke WS walewlole 
~----------- $----------4--------4--------------------- 
bsdmag | magazine | 129115 | t 

linuxmag | magazine | 129089 | f 

pentestmag | magazine | 129075 | £ 

public | magazine | 129047 | f£ 
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Listing 3 shows how to create each schema and how to in- 
sert the magazine table within each schema: please note 
that in the cREATE TABLE Command the fully qualified name 
of the table is specified. Doing introspection on the data- 
base (see Listing 4) does not provide the result readers 
would expect: only the bsdmag.magazine table is shown 
in the list; there is no mention of linuxmag.magazine and 
pentestmag.magazine, and the public.magazine table 
seems to have disappeared. The truth is that all the above 
tables are in place, as shown in Listing 5, but the system 
does not report them as “directly available”; to understand 
why, another concept related to schemas has to be intro- 
duced: the search path. 

The search path is a special PostgreSQL tunable 
that can be configured for each database user (and for 


TO 


each database session); its purpose is to instrument the 
searching within a set of schema locations for a non- 
qualified object name. Its usage is really similar to that of 
the PATH shell variable when searching for executables. 
By default, the schema search path is set to the tuple 
$user and public, which means that all the non-fully qual- 
ified objects will be searched first in a schema whose 
name is the same name of the database username that 
is currently running the session and then the public sche- 
ma. As readers can see, this is a very elegant trick to al- 
low each user to define a private table (or other kind of 
object) without having its name clash with instances of 
other users and without having to fully qualify it on each 
command (of course the object creation requires the fully 
qualified name). In all the examples shown in this article, 


Listing 8. Populating the three magazine tables 


INSERT INTO bsdmag.magazine (pk, id, month, issuedon, title) 
VALUES (1,’2Z012-01" ; 1; “Ol -Ol-Ol date, 
INSERT INTO bsdmag.magazine (pk, id, month, 
VALUES (27 ZU1i-t27) 12; PO OA: 
INSERT INTO bsdmag.magazine (pk, id, month, 
VALUES (6; 2OUL i i, 210) == be Ole args 


issuedon, title) 
date , ‘Rolling 
issuedon, title) 


date, 


‘FreeBSD: Get Up To Date’); 


‘Speed Daemons’); 


Your Own Kernel’); 


INSERT INTO linuxmag.magazine (pk, id, month, issuedon, title) 
VALUES (1,7 2012-01", 1, ‘2012-01-01’ ::date , 
INSERT INTO 1 
VALUES (2,/ 201 
INSERT INTO 1 
VALUES (3,/201 
INSERT INTO 1 
VALUES (4 ,/ 201 
INSERT INTO 
VALUES (5, / 201 


‘Understanding the Linux Kernel’); 


(pk, id, month, issuedon, title) 
“2012-04-00: dace, 


inuxmag.magazine 


i ee ‘Gnome and Linux’); 


(pk, id, month, issuedon, title) 
Ol l=Cl- Fil’ date, 


inuxmag.magazine 


Pel oles ‘Interview with A. Seigo’); 
(pk, id, month, issuedon, title) 


201-010 date, 


inuxmag.magazine 


ea0 aero ‘Compiling a kernel’); 


(pk, 1c, month, issuedon;, title) 
"2001 =U1=09" = date, 


linuxmag.magazine 


i eiOkc rch 


‘GNU Emacs’); 


SELECT count( m.pk) AS bsdmag issues 
FROM bsdmag.magazine m; 


bsdmag Ussuics 


SELECT count( m.pk) AS linuxmag issues 
FROM linuxmag.magazine m; 


FinuUxMag lesvies 


SELECT count( m.pk) AS pentestmag issues 
FROM pentestmag.magazine m; 


Pentestimag issues 
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the database connection is made via the user bsdmag, 
and therefore the suser special tag is substituted by the 
string bsdmag. The search_path can be inspected with 
the sHow command, and can be set with the set com- 
mand as shown in Listing 6. As readers can see, set- 
ting the search_path to the schemas /inuxmag and pub- 
lic changes the behaviour of the instrospection on the 
database. Listing 6 now shows the /inuxmag.magazine 
table as directly available. 

It is worth noting that even setting the search_path to the 
full list of schemas that include suser, linuxmag, pentest- 
mag and public does make more than one magazine table 
appear (while the system has one per each schema in the 
search_path). This is a design choice of PostgreSQL: in the 
case of name clashing the \a command shows only the first 
table that will be available in the search_path. This means 
that the other magazine tables are going to be hidden to the 
user until she qualifies them, and therefore the \d command 
reflects this state not showing a table that is not accessible 
without being fully qualified, even if the schema is in the 
search_path. PostgreSQL internally decides which tables 
are available in the case of name clashing using the pg_fta- 
ble_is_visible internal function, and users can inspect which 
magazine table is currently visible using \a with the query 
shown in Listing 7. Using the analogy of the PATH shell vari- 
able, the behaviour of the \a command is really similar to 


PostgreSQL: Schemas 


that of launching an executable using the PATH variable: 
the first executable hides the other in other directories. 

The above discussion emphasizes how the order of en- 
tries in the search_path is important, since it dictates which 
schemas have to be searched first and, in case of naming 
conflicts, which objects are being targeted and which are not. 

Finally, please take into account that the search_path can 
be configured for the whole cluster as explained in the Box 1. 

In order to complete the discussion about the same 
table within different schemas, Listing 8 shows a simple 
population of each different table and the count of each 
issues for the three schemas; please note that all the ta- 
bles exist independently from one another (Please note 
that, if all the tables the same structure, it could be worth 
using PostgreSQL inheritance (see previous articles on 
partitioning), but in order to keep the example simple and 
compact, the whole magazine table has been replicated 
within each schema). 


Using Schemas to Apply Privileges 

In order to demonstrate how schemas allows for a faster 
and easier set up of privileges, consider the case where 
two additional users have access to the database: /inux- 
mag_user and chief_editor (see Box 2 on how to create 
users). The former is a user strictly related to the /inux- 
mag set of database objects, while the latter is a user that 


Listing 9. Applying grant options using the schema facilities 


bsdmagdb=# GRANT USAGE ON SCHEMA bsdmag TO chief editor; 


// when connected as linuxmag user 

bsdmagdb=> SELECT * FROM bsdmag.magazine; 
ERROR: 
LINE 1: SELECT * FROM bsdmag.magazine; 


permission denied for schema bsdmag 


// when connected as chief editor 
bsdmagdb=> SELECT count( b.pk ) FROM bsdmag.magazine b; 


count 


bsdmagdb=# REVOKE ALL PRIVILEGES ON ALL TABLES IN SCHEMA bsdmag FROM linuxmag_ user; 
bsdmagdb=# REVOKE ALL PRIVILEGES ON ALL TABLES IN SCHEMA pentestmag FROM linuxmag_ user; 
bsdmagdb=# REVOKE ALL PRIVILEGES ON ALL TABLES IN SCHEMA bsdmag FROM chief editor; 
bsdmagdb=# REVOKE ALL PRIVILEGES ON ALL TABLES IN SCHEMA pentestmag FROM chief editor; 
bsdmagdb=# REVOKE ALL PRIVILEGES ON ALL TABLES IN SCHEMA linuxmag FROM chief editor; 
bsdmagdb=# GRANT SELECT ON ALL TABLES IN SCHEMA bsdmag TO elie. ech woe: 

bsdmagdb=# GRANT SELECT ON ALL TABLES IN SCHEMA pentestmag TO chief editor; 

bsdmagdb=# GRANT SELECT ON ALL TABLES IN SCHEMA linuxmag TO Clie Seemrou: 


bsdmagdb=% GRANT USAGE ON SCHEMA pentestmag TO chief editor; 
bsdmagdb=# GRANT USAGE ON SCHEMA linuxmag TO chief editor; 


www.bsdmag.org 
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should have access to all the sets for reading purposes 
(being chief_editor, not a database administrator). The se- 
curity policy therefore is as follows: 


¢ remove all privileges to all objects in the other sche- 
mas (bsdmag and pentestmag) from the linuxmag__ 
user. 

¢ provide privileges to chief_editor for all the schemas 
and all objects within the schema. 


While such policy can be implemented with a set of privi- 
leges on each object’s fully qualified name within a sche- 
ma, the Grant and rREvoKE command in PostgreSQL are 
schema aware and allow the DBA to quickly target all 
objects in a schema. Therefore, as shown in Listing 9, 
it is possible to specify an ALL TABLES IN SCHEMA 
target to have all the grants applied recursively. Please 
note that the sequence of commands in Listing 9 is not 
the only way of achieving the security policy described 
above, and that it is necessary to provide the usage priv- 
lege on the schemas in order to allow the chief_editor 
user to “walk” the schema (something similar to the file 
permission schema with the directory executable bit). 
The above example shows how to quickly apply custom 
privileges to a schema and a set of objects (tables) within it; 


Listing 10. An example backup of two of three schemas 


> po dump =n bsdmag =n linuxmag -U bsdmag bsdmagdb 


-- PostgreSQL database dump 


-- Name: bsdmag; Type: SCHEMA; Schema: -; Owner: bsdmag 


CREATE SCHEMA bsdmag; 
ALTER SCHEMA bsdmag OWNER TO bsdmag; 


-- Name: linuxmag; Type: SCHEMA; Schema: -; Owner: bsdmag 


CREATE SCHEMA linuxmag; 
ALTER SCHEMA linuxmag OWNER TO bsdmag; 


SET search path = bsdmag, pg catalog; 
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since the schemas used in this article are made by a table 
and a sequence, there is not a huge advantage in using this 
GRANT/ REVOKE syntax instead of addressing each fully quali- 
fied object. Nevertheless, in more complex deployment, it 
is quite common to have hundreds of tables within a single 
schema, and therefore having the capability of addressing 
a whole schema at once is a great time saver. 


Using a Schema to Manage Custom Backup/Restore 
Policies 

In the previous articles of this series readers have seen 
how pg dump (1) and pg restore(1) Can be used to make a 
cold backup (that is a consistent backup at a specific time). 
Since the above tools are schema aware, it is possible to 
use the commands to backup/restore a specific schema 
among those in the database. The -n option of pg dump (1) 
and pg restore(1) Can be used to specify one schema 
to backup; using multiple options allow the administrator 
to select multiple schemas at once, as shown in Listing 
19 where only the bsdmag and linuxmag are going to be 
dumped. Of course it is important that the user that exe- 
cutes the dump have the privileges to access all objects in 
the selected schemas. It is worth noting how the dump per- 
forms the initial setup of the schemas and the search path 
for accessing objects during the restore phase. 


Using Schema for Customization of Configuration 
As described above, changing the search_path for a user 
allows the overriding of some database objects that have 
the same name. This allows for a per-user configuration and 
customization, since each user could be “pushed” to search 
for a particular object into a specific schema. To better un- 
derstand, consider the Listing 11, that defines three ver- 
sions of the same stored procedure download_url that, giv- 
en the primary key of a magazine tuple returns a download 
URL for an issue. Each version of the function goes into one 
of the three schemas and returns a prefix that changes de- 
pending on the magazine it belongs to (Again, this is not the 
ideal design, but is used only to explain the schema facility). 
As shown in Listing 12, having different search_path allows 
a user to “see” different results, and therefore this can be 
used as a trick for differentiating users’ profiles. 

Suppose the /inuxmag_user user has to be fully custom- 
ized so that when calling the download url () stored pro- 
cedure the linuxmag.download_url() Is effectively called; 
there are two ways of achieving this: 


¢ setting the search_path of the user so that the linux- 
mag schema is the first entry; 

¢ configure a linuxmag schema with a stored procedure 
that WrapsS linuxmag.download_ url(). 
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Listing 11. Three procedure placed each in a different schema 


CREATE OR REPLACE FUNCTION bsdmag.download_ url ( 
magazine pk integer ) 
RETURNS text 
AS 
SBODYS 
DECLARE 
magazine id text; 


BEGIN 


-- get the magazine id 
SELECT id 

INTO magazine id 

FROM magazine 

WHERE pk = magazine pk; 


IF magazine id IS NULL THEN 
RETURN ‘’ ; 
END IF; 
RAISE LOG ‘bsdmag.download url()’; 
-- this is the part that changes depending on 
Evens chema 
RETURN ‘http://bsdmag.org/download/’ || 


MievefeAsives abel || | -yeichay = 


END ; 
SBODYS 
LANGUAGE plpgsql; 


CREATE OR REPLACE FUNCTION linuxmag.download url ( 
magazine pk integer ) 
RETURNS text 
AS 
SBODYS 
DECLARE 
magazine id text; 


BEGIN 


-- get the magazine id 
SELECT id 

INTO magazine id 

FROM magazine 

WHERE pk = magazine pk; 


IF magazine id IS NULL THEN 
RETURN \’ ; 


END IF; 


RAISE LOG ‘linuxmag.download url()’; 
-- this is the part that changes depending on 
ine sclveiel 
RETURN ‘http://linuxmag.org/download/’ || 


rileiopewaaans mel |f\|  ajorele 


END ; 
SBODYS 
LANGUAGE plpgsql; 


CREATE OR REPLACE FUNCTION pentestmag.download_ url ( 
magazine pk integer ) 
RETURNS text 
AS 
SBODYS 
DECLARE 
Magazine 1d text; 


BEGIN 


-- get the magazine id 
SELECT id 

INTO magazine id 

FROM magazine 

WHERE pk = magazine pk; 


IF magazine id IS NULL THEN 
RETURN °’ ; 
END IF; 
RAISE LOG ‘pentestmag.download url()’; 
-- this is the part that changes depending on 
ine SC Meine! 
RETURN ‘http://pentestmag.org/download/’ || 


Magazine wee Cn a 


END ; 
SBODYS 
LANGUAGE plpgsql; 
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The first way is left as an exercise for the reader, while 
the latter is shown in Listing 13: first a schema with the 
same name of the user has to be created, and then a 
wrapper function is placed in such schema. It is even 
possible to move the stored procedure from its origi- 
nal schema to the user's, but this could make it impos- 
sible for other users to access the same procedure (in 
this case, security restrictions on the per-user schema 
are enforced). Finally, an alias to the right magazine ta- 
ble has to be set up, and this is done via a view. 

As shown in Listing 14, calling the download url () proce- 
dure as the /inuxmag_user now redirects to the 1inuxmag. 


download_url() procedure in the view that in turn uses the 
linuxmag.magazine table. As shown, the per-user schema 
configuration is a very powerful feature that requires a lit- 
tle extra effort for a correct set up, but can turn out to be 
a fundamental capability to allow portability. It is clear that 
all the above function definitions and per-schema object 
set up can be automated using scripting and stored pro- 
cedures (see previous articles), reducing the DBA load. 


Operating on a Whole Schema 
A DBA can perform other interesting operations on a 
schema and all its contained objects at once, the most 


Listing 12. Changing the schema search_path allows a user to get 
different behaviours 


bsdmagdb=# SELECT download url( 1 ); 
LOG: bsdmag.download url() 
download _ url 


http: //bsdmag.org/download/2012-01.pdf 


bsdmagdb=# SET search path TO pentestmag, linuxmag, bsdmag 
pjewledkikey 
bsdmagdb=# SELECT download url( 1 ); 
LOG: pentestmag.download url() 
download _ url 


http://pentestmag.org/download/2012-07.pdf 


Listing 13. Setting up objects for a complete per-user customization 


CREATE SCHEMA linuxmag user; 


-- remove all privileges to all other users 
REVOKE ALL PRIVILEGES ON SCHEMA linuxmag user FROM 
PUBLIC; 


== Grant all privileges to the running user 
GRANT ALL PRIVILEGES ON SCHEMA linuxmag user TO 


linuxmag user; 


== Grant Usage for Ehe schema targel Of Lhe functions 
GRANT USAGE ON SCHEMA linuxmag TO linuxmag_ user; 
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA linuxmag TO 


linuxmag user; 


-- create a wrapper function 
CREATE OR REPLACE FUNCTION linuxmag_user.download_ url ( 
magazine pk integer ) 


RETURNS text 


AS 

SBODYS 

DECLARE 

BEGIN 
RAISE LOG ‘linuxmag_user.download url()’; 
RETURN linuxmag.download url( magazine pk ); 


END ; 
SBODYS 
LANGUAGE plpgsql; 


-- create a wrapper view for the magazine table 
CREATE OR REPLACE VIEW linuxmag user.magazine 
AS 

SELECT * 


FROM linuxmag.magazine; 


Listing 14. Example of calling the download_urlI() procedure as 
linuxmag_user user 


bsdmagdb=> SELECT current_user; 


current user 


linuxmag user 


bsdmagdb=> SELECT download _url( 1 ); 


LOG: linuxmag_user.download url () 

LOG: linuxmag.download url () 

CONTEXT: PL/pgSQL function “download url” line 5 at RET 
URN 

LOG: duration: 5.670 ms statement: SELECT download_ 
Gligalat (ied ES) 


download _ url 


htto:// linuxmagvorg/download/ 2012-01, dt 
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common being dropping the whole schema and renaming 
the schema. The prop scHEMA Command is used to drop 
a schema; since the schema can contain different objects 
the database prevents accidental schema deletion by in- 
forming the user about dependencies, as shown in the top 
half of Listing 15. In order to recursively drop a schema 
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and all its content it is required to use the command prop 
SCHEMA ... CASCADE, aS shown in the bottom half of Listing 
15. The system will inform the user about the objects that 
are going to be dropped due to the cascade option. 

To rename a schema the special command a.ter 
SCHEMA. ..RENAME Can be used, as shown in Listing 16. The 


Listing 15. Dropping the linuxmag schema 


bsdmagdb=# DROP SCHEMA linuxmag; 
ERROR: 
DE WA Tin; 


HINT: Use DROP ... 
bsdmagdb=# DROP SCHEMA linuxmag CASCADE; 
NOTICE: drop cascades to 3 other objects 
DETAIL: drop cascades to table linuxmag.magazine 

drop cascades to view ]linuxmag_user.magazine 

drop cascades to function linuxmag.download url (integer) 


DROP SCHEMA 


Listing 16. Renaming a schema and changing the owner 


Listing 17. Creation and inspection of a temporary table 


bsdmagdb=> \d temp table 
Table “pg temp 1.temp table” 


Column | Type | Modifiers 
ee eee Efe oases ol ere ew ee ee 
pk | integer | not null 
title | text | 

Indexes: 


“temp table pkey” PRIMARY KEY, btree (pk) 


bsdmagdb=> SELECT n.nspname, -- schema name 
c.relname, -- relation name 
el feulicl, == iselleve moi © ite) 


FROM pg class c 

LEFT JOIN pg catalog.pg namespace n 
ON n.oid = c.relnamespace 

WHERE c.relname = ‘temp table’ 

ORDER BY n.nspname; 

nspname | relname | ontel seo table vis visible 

----------- 4------------4--------4}--------------------- 


eg _tene l | cemeveable || 129142 |Fr 


cannot drop schema linuxmag because other objects depend on it 
table linuxmag.magazine depends on schema linuxmag 

view linuxmag user.magazine depends on table linuxmag.magazine 
function linuxmag.download url(integer) depends on schema linuxmag 


CASCADE to drop the dependent objects too. 


bsdmagdb=# ALTER SCHEMA pentestmag RENAME TO archived pentestmag; 
bsdmagdb=# ALTER SCHEMA archived pentestmag OWNER TO linuxmag_ user; 


bsdmagdb=> CREATE TEMPORARY TABLE temp table( pk integer NOT NULL PRIMARY KEY, title text ); 


jog) eciccillerg sie teclolliS is i isslelel( we ceukcl ))) == ils clive Sieldiciie iar elie. Secuccla po eicla” 
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ALTER SCHEMA COmmand can also be used to change the 
schema owner. It is worth noting that changing the owner 
of a schema will not affect the ownership of contained ob- 
jects; they will keep their previous owner. To do it mas- 
sively a script that iterates on pg_class or pg_tables (in 
the case of only tables) and issues more alter commands 
has to be used. 


Box 1. Setting the search_path for all clients 
The PostgreSQL configuration file postgresql.conf contains a 
definition of the variable search_path that can be used to set 
the search_path for each client that connects to the cluster. 
Specify the string that defines the search path, as used in the 
SET command (with quotes), to make the clients receive the 
path as their initial search path: 


Search path = ““Suser”, public’ 


It is interesting to note that this change affects the whole clu- 
ster, not a single database, and therefore for very specific set- 
tings it is better to work on the single client connection. 


Box 2. Creating database users 

Database users can be created using the CREATE ROLE com- 
mand, launched by a database superuser. The command pro- 
vides many options, therefore the following is just an example 
of how to quickly create a user: 


bsdmagdb=— CREATE ROLE chier ediror WiIh NOGIN PASSWORD: “chick”; 


Box 3. Moving objects across schemas 
PostgreSQL allows a database object to be moved across dif- 
ferent schemas, of course assuming the user that moves the 
objects have the rights to do the delete/insert operation in 
the source/target schemas. The schema migration is done us- 
ing the ALTER...SET SCHEMA commands, available for any 
kind of object that can be enclosed into a schema (e.g., tables, 
stored procedures, etc.). AS an example, to move the table pu- 
blic.my_table from the schema public to the schema my_sche- 
ma it is possible to use the following command: 


bsdmagdb—; ALTER TABLE public my cable sil SCHEMA my sschema, 


An exception to the above is for temporary objects, like tem- 
porary tables, that cannot be moved out from their schema: 


bsdmagdb=# ALTER TABLE temp table SET SCHEMA linuxmag; 


ERROR: cannot move objects into or out of temporary schemas 


On The Web 

¢ PostgreSQL official Web Site: http:/www.postgresql.org 
ITPUG official Web Site: http://www.itpug.org 
Oddity with \d and pg_table_is_visible: http://archives. 
postgresql.org/pgsq!-hackers/2007-09/msg00205.php 
GitHub Repository containing the source code of the ex- 
amples: https://github.com/fluca1978/fluca1978-pg-utils 
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Temporary Tables 

Temporary tables are a feature that allows a database to 
contain a table that will not persist on disk at any time, 
and therefore will not be recoverable (it is not written at 
all in the WAL logs) and cannot be backed up. The idea 
is to define a table from scratch to store in it some vola- 
tile location for testing or to create a materialized set of 
data to speed up later computations. Temporary tables 
are not strictly related to schemas, but their implementa- 
tion is based on schemas. When a user creates a tem- 
porary table using the cREATE TEMPORARY TABLE Command 
(as shown in Listing 17), the table is placed in a special 
schema named after the progressive connection num- 
ber pg_temp_X (being X the number of the connection to 
the cluster). AS shown in Listing 15, the temporary table 
temp_table is qualified by the name pg temp_1.temp_ta- 
ble. The fully qualified table is defined as any other table, 
and therefore is available to other users and sessions, at 
least until is destroyed. The search_path of the user that 
has defined the table is not changed, however the user is 
able to access the table even using the simple name. This 
is due to the pg table is visible () internal function re- 
turning true for each temporary table defined by the user 
himself (see bottom half of Listing 17). The same is not 
true for all other users, that are required to access the 
table using the fully qualified name. 

It is worth noting that temporary schemas pg temp_xX 
are sealed: objects cannot be moved into or out of them 
(see Box 3). However it is possible to create objects into 
a pg_temp_X schema, even if such kind of objects will be 
destroyed when the client disconnects. 


Summary and Coming Next 

This article introduced the concept of schema, a very 
powerful abstraction that allows DBAs to organize data- 
base objects into coherent and interrelated packages. In 
the next article the management of users, groups and per- 
missions within PostgreSQL will be shown. 
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What you will learn... 
¢ In this paper we will learn to setup and manage a Small Business 
DNS/DHCP server. 


one needs some basic parameters, which identi- 
fies it uniquely within the network itself, namely: 

IP address — subnet mask (netmask) — gateway ad- 
dress — a DNS server to resolve domain names (Domain 
Name System). 

A DNS Server "translates" the domain name (for exam- 
ple pippo.com) to an IP addresses (such as 192.168.10.1), 
and it is the only one that allows you to uniquely identify 
the machines within the networks, including the Internet. 

A DNS server configuration is usually only used in large 
networks, and almost never in small LANs, in which the 


7 or connecting a PC to a LAN (Local Area Network) 


Figure 1. A typical Small Business LAN Scenario 
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What you should know... 
¢ Basic BSD Networking Setup and basic Networking structure knowl- 
edge. 


resolution service domain names relies on an external 
server. In this article we will see how to set the automat- 
ic configuration of network parameters on each machine 
connected to the LAN, including the resolution of domain 
names to the hosts in it. What you will get will be a service 
able to ensure: 


¢ ADNS configuration of machines "behind" the firewall 
simple and independent by a DNS provider 

¢ Timeout immediate for clients in the absence of the 
internet 

¢ Names of local machines centralized on the firewall's 
file /etc/hosts automatically propagated. 

¢ DHCP service switch with DHCP leases static and 
dynamic IP ranges and multiple. 

¢ Caching internet addresses (A records and AAAA re- 
cords and PTR records) with improved network per- 
formance. 

¢ Support for MX and SRV records type and ability to 
provide the MX record for some or all machines on 
the local network, including the resolution of domain 
names to the hosts in it. 


For the complete setup of a LAN DNS/DHCP service 
one usually needs BIND (Berkeley Internet Name Do- 
main) and ISC-DHCPD (Dynamic Host Configuration 
Protocol), both available at www.isc.org. BIND is "the" 
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DNS server used by many ISPs and Domain Register, 
which, however, has the flaw that its configuration is not 
simple, is based mainly on the creation of cryptic text 
files for different areas (hostname, domain, or sections 
of domains) and has a complex base configuration, also 
for simple scenarios (Figure 2). 

DHCP, instead, is composed of a client / server system 
and is responsible for the automatic configuration of the 
network parameters. In practice, DHCP clients (PCs con- 
nected to the network) send the request to the server to 
get configuration parameters. 

On the other hand, the DHCP server receives the re- 
quest and, based on the MAC address (hardware address 


Figure 2. Bind + ISC-DHCPD Operating Diagram 


Figure 3. Dnsmasq Operating Diagram 
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The BSD Certification Group Inc. 
(BSDCG) is a non-profit organization 
committed to creating and 
maintaining a global certification 
standard for system administration 
on BSD based operating systems. 


@ WHAT CERTIFICATIONS ARE AVAILABLE? 


BSDA: Entry-level certification suited for candidates 
with a general Unix background and at least six months of 
experience with BSD systems. 


BSDP: Advanced certification for senior system administrators 
with at least three years of experience on BSD systems. 
Successful BSDP candidates are able to demonstrate 

strong to expert skills in BSD Unix system administration. 


@ WHERE CANIGET CERTIFIED? 


We're pleased to announce that after 7 months of 
negotiations and the work required to make the exam 
available in a computer based format, that the BSDA 
exam is now available at several hundred testing centers 
around the world. Paper based BSDA exams cost $75 USD. 
Computer based BSDA exams cost $150 USD. The price of 
the BSDP exams are yet to be determined. 


Payments are made through our registration website: 
https://register.bsdcertification.org//register/payment 


@ WHERE CAN I GET MORE INFORMATION? 


More information and links to our mailing lists, LinkedIn 
groups, and Facebook group are available at our website: 
http://www.6sdcertification.org 


Registration for upcoming exam events is available at our 
registration website: 
https://register.bsdcertification.org//register/get-a-bsdcg-id 
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that is unique to each network adapter) of the client, re- 
sponds by providing the network parameters necessary to 
use the local network and the Internet. 

The local IP address can also be uniquely associated 
with a MAC address. In this way, the user (the machine) 
will always have the same address instead of a random- 
ly chosen among those available. In addition, the DHCP 
server can be configured to communicate to the BIND 
hostname and address of the machine that provided the 
connection parameters in order to automatically update 
the zone file on the local network. 


Dnsmasq, the Alternative for Small Offices 
Dnsmasgq, the software that we will manage, is basically 
a mini DNS server that can resolve the names of computers 
on the local network and at the same time provide a DH- 
CP service, although it is not suitable when you need ad- 
vanced features such as DNS configuration master / slave 
or to manage a very large number of "Zones" (Figure 3). 


Pre-installation Steps 

We must do some presets before starting the installation 
of the Dnsmasgq server, assuming the same pc that hosts 
it will act as a gateway firewall for our LAN. 


On FreeBSD 
We must correctly setup /etc/rc.conf to have an operating 
network setup, in particular we need to configure the LAN 


Listing 1. A basic /etc/rc.conf file for a classic Dnsmasq Server FW 


#t## rc.conf 

# WAN static connection 

i peeconio, <lN0—"iner 1020702254 netmask 259-255. 299.0" 
# WAN dhcp connection 

ifconfig xl0="dhcp” 


7 LAN connect Lon 


Hie Onin nieteh 2 los ictitas S699. 200.2905" 


# Default gateway 
# Set the gateway for static connection 


ip oleunenollki@ieeone ic“ ANU Os Way 1 


# Enable ip forward 
gateway enable="YES” 


# Hostname 
hostname="fw.bsdmag.lan” 


to? end ceecont 
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network interface with a static IP, as shown in Llisting 1. 
Set the DNS in /etc/resolv.conf: 
nameserver LZel 660 el 


nameserver 10.0.0,1 # for static connection 


On OpenBSD 

We must correctly setup the system to have an oper- 
ating network setup, in particular we need to config- 
ure the LAN network interface with a static IP, as shown 
below: 


/etc/hostname.xl1 


inet. Di ZsLOs Usd 252200625960 
# LAN NETWORK SETUP 


LiZslGe.0s259 


/etc/hostname.xl0 
dhcp # WAN DHCP NETWORK SETUP 


We must enable port forwarding by uncommenting this 
line InN /etc/sysctl.conf: 


#net.inet.ip.forwarding=1 


Setup the OpenBSD box's default gateway editing /etc/ 


mygate. 


Lie los Ued 


Setup the OpenBSD box's hostname editing /etc/ 


myname. 
fw.bsdmag.lan 
Setup the /etc/rc.conf.local services 


dhcpd_ flags="x10" 
#pf=NO 


Set the DNS in /etc/resolv.conf: 


172.16/0.1 
<YOUR_ISP_NAMESERVER> 


nameserver 


nameserver 


Setup via Source Code or Using Packages 

One may install Dnsmasq on any compatible Unix plat- 
form. Just choose whether to use the installation from 
source (the latest release is dnsmasq-2.63) or from 
a package. 
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Dnsmasq on OpenBSD 5.1 - 5.2 

On OpenBSD 5.1 we need to do a little hack: First navi- 

gate to the rc.d directory and download the start script: 

Ga. jetc/re,.d 

curl -o dnsmasqd \ http://ftp.openbsd.org/ports/net/ 
dnsmasq/pkg/dnsmasq.rc. 

Setup like this line 5: 

daemon="/usr/local/sbin/dnsmasq" 


Start the service using: 


/etc/rc.d/dnsmasgqd start 


Listing 2. The classic /etc/resolv.conf file of aDnsmasq Server 


# OpenDNS DNS SERVERS 
nameserver 127.0.0.1 
nameserver 208.67.222.222 
nameserver 208.67.220.220 
# DNS SERVERS GOOGLE 
nameserver 8.8.8.8 


nameserver 8.8.4.4 


Listing 3. A simple dnsmasq.conf file 


no-dhcp-interface = xl0 

addn-hosts = /etc/dnsmasq-hosts 

INO 1ANS, Nes 

local = /bsdmag.lan/ 

interface = xll 

expand-hosts 

domain = bsdmag.lan 

CINCO =ealiGie= = oli ZkG 0 2 lei GR Ola OO ehh 
CNCO=CPEION = OpDulon: seouker 777 lo2 tial 
aiicp=optiron = 44, 172.16.0.1 

cllacioejein sein 8 85 I Ie 0) IL 

dhcp-option = 46.8 

dhcp-option = 47 

Gmeép=oprEvon = 6,172 bold! 

mx=host = mail bsdmagslan, 50 

mx-target = mail.bsdmag.lan 

localmx 

log-queries 


log=ciie. 


www.bsdmag.org 


If it says it starts In rc.conf.local add: 
pkg scripts="dnsmasqd” 


On OpenBSD 5.2 to get the software last release one 
may install from sources: 


# cd / tmp 

# wget-c http://www.thekelleys.org.uk/dnsmasq/ 
dnsmasq-2.63..tarsgzZ 

# tar-xvzf dnsmasq-2.63.tar.gz 

# cd dnsmasq-2.63 


# make install 


lf one dislikes to install from sources, it's possible to use 
the packaged version, but it's not updated. 


# pkg add -r -v dnsmasq 


Dnsmasq on FreeBSD 
One can install Dnsmasq by using the ports collection: 


S$ cd /usr/ports/dns/dnsmasq/ 
S su 


# make install clean 
or by using pkg _ add with the command 
# pkg add -r -v dnsmasq 


The Dnsmasgq script will be installed in /usr/local/etc/ 
rc.d, and to get it to start at boot time, add this line to 


fete/eescont. 

dnsmasq enable="YES" 
Then start Dnsmasa: 
S su 


# /usr/local/etc/rce.d/dnsmasq start 


Basic LAN Name Configuration 

Dnsmasq normally uses only the system file /etc/hosts On 
the PC running the service, associating the following names 
to IP addresses, but here we will use a static address file 
just by putting them in a new file /etc/dnsmasq-hosts (aS 
specified by the parameter addn-hosts) in the form without 
a domain. We do not use the default /etc/nosts file in this 
scenario for preventing DNS server to resolve the "private" 
names that one may put here, for example "localhost". 
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# cat /etc/dnsmasg-hosts 


172.16.10.1 proxyserver firewall 
172.16.10.2 vpnserver 


As you can see from this extract of the hosts file, a ful- 
ly qualified domain name or more “short names” can be 
associated with each IP. 


Basic configuration of external DNS server 

To resolve names outside your local network, Dnsmasq 
uses the DNS servers in /etc/resolv.conf, which is struc- 
tured as follows: Listing 2. 


Configuring Services 

Now that the external DNS servers is configured, it's time 
to setup the file dnsmasq.coné. This file in OpenBSD setup 
is stored in /etc folder, in FreeBSD setup is in /usr/local/ 
etc folder. 


EXTERNAL ONS SERVER 


MASTER ONS 1 
|] ——| —~ | 


FW BSD 
xl0 dhcp 
xl1 172.16.0.1/24 

xl2 172.16.10.1/24 


DnsMasq Server 


ISP mot em/router 


Figure 5. A possible Dnsmasq LAN+DMZ Scenario 
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The file is well commented. We may set the following 
parameters for a starting configuration, but We must re- 
member to setup network interfaces and IP range con- 
sistent with our hardware and network architecture, es- 
pecially lines as “interfaces”, “no-dhcp-interface” and 
“dhcp-range’: Listing 3. 

And we will explain all the configuration sets: 


¢ local =/ .... / adds declarations for forward and re- 
verse DNS queries 

¢ expand-hosts add the domain to simple names 

¢ domain =... specifies DNS domains for the DHCP 
server 

¢ no-dhcp-interface = ... Which interface set on the 
server must not listen to 

¢ dhcp-range = ... sets up the DH CP ip pool with a de- 
fault lease time 

¢ dhcp-option = option: router, .... sets up the LAN gate- 
way 

¢ log-queries logs requests dns 

¢ log-dhcp logs dhcp requests 

e addn-hosts = _ /etc/dnsmasq-hosts_ file speci- 
fies the alternative hosts for the resolution of local 
machines 

¢ no-hosts specify to not use the hosts file for name 
resolution 

¢ dhcp-option = 44,172.16.0.1 Set NetBlOS-over-TCP/ 
IP nameservers aka WINS servers 

¢ dhcp-option = 45,172.16.0.1 netbios datagram distri- 
bution server 

¢ dhcp-option = 46.8 netbios node type 

¢ dhcp-option = 47 empty netbios scope. 


¢ dhcp-option = 6,172.16.0.1 setup the lan dns 
server 
¢ mx-host = mail.osdmag.lan, 50 useful for direct- 


ing mail from systems on a LAN to a central ser- 
ver 

¢ mx-target = mail.obsdmag.lan specify the default tar- 
get for the MX record returned by Dnsmasq 

¢ localmx return an MX record pointing to the host Giv- 
en by mx-target for each local machine 


To see all the parameters "dhcp-options" is possible run 
the command: 


# dnsmasgq - help dhcp 


For explanations on the individual parameters, refer to 
the online help 


# man dnsmasq 
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As we have seen, to enable the DHCP server is neces- 
sary to define at least one of its essential directives such 
as, for example, "dhcp-range": 


no-dhcp-interface=x10 
dhcp=-range=1/2.16.0.2,172.16.0.200, 12h 
dhcp-option=option:router,172.16.0.1 


The first line excludes a network interface from DHCP, 
which, however, will continue to be valid for DNS re- 
quests. The second line defines the range of addresses 
to be assigned dynamically. 


Advanced Configuration 

The default behavior is to communicate to clients the Dns- 
masq server IP as a common gateway and DHCP/DNS 
services provider. However, the computer running Dns- 
masq is not always a gateway, but we can force the cor- 
rect gw adress with the directive 


dhep-option=option: router, 1/2.,16.0.1. 


In a more complex scenario we may have two internal 
network interfaces (xl1 for LAN and xl2 for DMZ ), and 
one external (xl0 connected to the Internet), to serve a 
small business with a more complex topology that pub- 
lish service to the Internet (Figure 5). 

For a complete list of options is necessary to refer to the 
online dnsmasq manual: http:/leaf.sourceforge.net/doc/ 
man/dnsmasq.8.html (Listing 4). 

Let's now explain the rows of the configuration. If we 
want to associate fixed addresses to some machines, we 
record their MAC address and add one of the following 
directives: 
172.16.10.40 


dhcep-host = 00: co: 77:26:26: serverl, 


except-interface= ... 


Listing 4. A more complex configuration file for Dnsmasq 


no-dhcp-interface = x10 
except-interface=x10 

addn-hosts = /etc/dnsmasq-hosts 
BOGUS =p ally, 
resolv-file=/etc/resolv.conf.dnsmasq 
INO ave ne ss 

local = /bsdmag.net/ 

expand-hosts 


domain = bsdmag.net 


# LAN DHCP DNS Service listening on interface xll 
interface = xll 
Oico=mange- =a 2G rz li ko 0 200 ak Zin 
dhice=-op lon = al oOprlron: scoulwer a li2 to a0. 
dhice-option =I 44 17/2216 .0.1 
dhep=optton =< 745,172.16 205 | 


dhcp-option =xl1,46.8 
dhcp-option =xl1,47 
dhep=opemon =e igre 0.1 


# DMZ DHCP DNS Service listening on interface x12 
interface = xl2 

dhce=rande==xi2 ii le 2 i 2 lo 007 T2i 
dhep=opEton =<I7 operon, Tovlkery. 2 G0 al 
dhce=-optionm =xl27 447 di iG. ho 

dhep=oprron = aa 2 Gl Or 


dhcp-option =xl2,46.8 


dhcp-option =xl2,47 


Ghep=Opieton w= sclA7 6 a7 she aOr al 


# Generic stuff 

mx-host = mail.bsdmag.net, 50 

mx-target = mail.bsdmag.net 

localmx 

log-queries 

log=dier 

cache-size=2048 
log-facility=/var/log/dnsmasg/dnsmasq.log 


dhcp-leasefile=/var/log/dnsmasg/dnsmasq.leases 


? Steere uDHEP ogra ims: 
chep=nost = 002 -eGn, 7/2262 26: serverL, liye 10.40 
Ghiceehos: = 02.66: 56. lo237. servers bi Zak koa 


Listing 5. A /etc/resolv.conf file for the advanced Dnsmasq setup 


search bsdmag.lan 
nameserver 127.0.0.1 
nameserver 8.8.8.8 


nameserver 8.8.4.4 


www.bsdmag.org 
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We do not listen on the specified interface. 


bogus-priv 
We do not propagate the addresses referred to areas that 
are not rotated and all the reverse lookup for the private 
subnet not present in /etc/dnsmasq-hosts or DHCP lease 
obtain "host not found" instead of being forwarded to the 
external dns. 


resolv-file=... 
we get all the IP addresses of the upstream nameservers 
from <file> instead of using / etc / resolv.conf. 


cache-size=... 
It's the size of Dnsmasgq's cache. The default is 150 
names. Setting the cache size to zero disables caching. 


log-facility=... 

set the facility to which Dnsmasq will send various logs, 
if the variable contains at least one '/' character is as- 
sumed to redirect the output to a file instead of syslog. 


dhcp-leasefile=... 
this is the file where Dnsmasgq file keeps track of ip deliv- 
ered to clients. 

lf you have multiple interfaces that offer DNS / DHCP 
must specify the prefix “ifname,” just before the network 
parameters in the Directive "dhcp-option", as in the previ- 
ous file. 

Now we modify our /etc/resolv.conf 


# emacs /etc/resolv.conf 


which must contain the DNS server queries, ie, itself and 
secondly the DNS provider which will make the cache 
(for example, 8.8.8.8) (Listing 5). 


Logging Analysis 
As shown there are four main parameters to configure 
logs: 


log-queries 
Log=dhicp 
log-facility=/var/log/dnsmasg/dnsmasq.log 


dhcp-leasefile=/var/log/dnsmasg/dnsmasq. leases 


But if we experience strange behavior we'll be able to 
analyze DHCP packets for monitoring or debugging pur- 
poses by using tcpdump and dhcpdump programs. The 
last one provides a tool for visualization of DHCP pack- 
ets for analyzing DHCP server responses in tcpdump 
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style. We may compare the output of tcpdump and dhcp- 
dump commands, but here we will only explain the syn- 
tax, to let you see the differences. 

Here we use tcpdump to capture DHCP output 


# tcpdump -lenx -i xll =<s 1500 port bootps or port bootpc 
and here dhcpdump 


#dhcpdump -i xll 


What do you think about the differences?? 


Conclusions 

A real example of small LAN business network are the so 
called “SoHo” (single office/nhome office SOHO), namely 
a category of businesses that has 1 to 10 employees, but 
this is only the staring point. In fact, there are examples of 
deployable environment for Dnsmasg configurations used 
for more than 1000 hosts. On the other side of the coin 
there are still some limitations, such as a very basic sup- 
port for IPv6 router advertisements for DHCPv6 to work 
and the inability to serve many zone files (many domains), 
but this project brought us many surprises in time and will 
only get better. Knowing the strengths and limits of this 
daemon, a network administrator can now decide whether 
to install Dnsmasq. 
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SECURITY 


Hardening FreeBSD 


with TrustedBSD and Mandatory Access Controls (Part 4) 


Most system administrators understand the need to lock down 
permissions for files and applications. In addition to these 
configuration options on FreeBSD, there are features provided by 
TrustedBSD that add additional layers of specific security controls 
to fine tune the operating system for multilevel security. 


What you will learn... 
¢ Configuration of the mac_seeotheruids module. 


tensions have been included with the default in- 

Stall of the operating system. By default, this func- 
tionality is disabled and requires support to be compiled 
in or kernel modules to be loaded at boot time. For the 
purpose of this article, support will be loaded in with ker- 
nel modules already available with FreeBSD 9. Part 4 of 
the TrustedBSD series will cover the basic configuration 
of the mac_seeotheruids module. 


S ince version 5.0 of FreeBSD, the TrustedBSD ex- 


What you should know... 
- Basic FreeBSD knowledge to navigate the command line 
¢ Familiarity with loader.conf to enable kernel modules at boot 


Warning 
Incorrect MAC settings can cause even the root user to not 
be able to login to the system. Be sure to run these tests on 
a VM or test machine to avoid any issues with production 
systems. This article assumes that a fresh install of Free- 
BSD 9.0 has been performed before continuing. 

As in the previous articles, a certain set of users will 
help to illustrate how to use mandatory access controls 
(MAC). For the mac_seeotheruids module, the purpose is 


Listing 1. Users setup on FreeBSD 


PwWoulsetr add —netisenia —< sy bam, coh —m 
Di Use @ ddd ane eek? =e 7 ban, Cali. ml 
pw user add -n user3 -s /bin/csh -m 
pw user mod -g user-reg -n userl 


pw user mod -g user-reg -n user2 


S$ + S$ S$ SF SF 


passwd userl 

Changing local password for userl 
New Password: 

Retype New Password: 

# passwd user2 

Changing local password for user2 
New Password: 

Retype New Password: 


# passwd user3 


Changing local password for user3 
New Password: 

Retype New Password: 

# groups userl 

userl user-reg 

# groups user2 

user2 user-reg 

# groups user3 

user3 

i) SVGCIL secwliciicy scl see Crclieie blibels 
SeOlioley lsc SoS Cues bavi s Il 

i? SOPSOUIL Sereqhemiey losol, Siete. er claeie ie auels 


Secu pin WSdnocemomle GaCihds a. 
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to restrict the ability to view the running processes of other and groups to see the running processes of other users 
users on the system. Listing 1 shows a basic setup forthe and groups on the system. The mac seeotheruids mod- 
required users for this article. ule adds additional features beyond the normal security. 

The sysctl values for security.bsd. see other uids and bsd see orher uids and security.bsd. see other gids 
security.bsd.see other gids aresetto1toallowallusers sysctl values on the system. Listing 2 shows how to load 


Figure 2. Three separate window panes in tmux for the root, user! 
Figure 1. Using tmux and multiple window panes and user2 users 


Listing 2. Loading the mac_seeotheruids module on system startup 


# echo ‘mac seeotheruids load="YES”’ >> /boot/loader.conf 
# echo ‘security.mac.seeotheruids.enabled=0'’ >> /etc/sysctl.conf 
# reboot 


(The sysctl values change the default values which enables the module.) 


Listing 3. /nstalling tmux 


# pkg add -r tmux 
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9.0-release/Latest/tmux.tbz... Done. 
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9.0-release/All/libevent-1.4.14b 2.tbz... Done. 


Listing 4. Start a loop in the user2 window then navigate to the user! window and run “ps -aux|grep user2” to view the running processes 


(Type the following in the user2 window:) 
%echo “while 1; echo ‘user2’ && sleep 1; end” | csh -f 
user2 


user? 


(Use Ctrl-b and the arrow keys to go to the userl window and type the following) 


Sey) / 


Sps aux | grep 


userl 88500 OF Oras SOG 304 2 RGe Paes 0:00.00 grep user2 
“wOOE 84297 O20) 6G) 42 SiG beZ8 oe. 2 6:45PM C200 Ol Tsnh = userZ 
user2 84293 O20 aa 2388 Sree S 6:45PM O20 v=cie iesin) 
user2 88467 O20) lao leo l2 1804 Sot ASME, Oes00 2 OM erste 
user2 88498 O20 Oey SO1G 780 Som 7 APM 0:00.00 sleep 1 


% 


(userl can see user2 processes) 
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Listing 5. Set the sysctl value for security.bsd.see_other_uids to 0, then try to view the processes of user2 with user. (Note: the while loop in 
the user2 window should still be echoing “user2") 


(In the root window, type the following) 
fesyocel scCurivy.bs@. sco OLNen uilds—0 
Secueihny Wsdnoce sommes. a l= 2a) 


# ps aux | grep ‘“*user2’ 


COOE ol 049 CaO 10m 3 oC 308 1 Rt 7 ee 0:00.00 grep user2 
root 84292 ORO Oe Al OG 1828 ote 6:45PM OP 0 Ost = swser7Z 
user2 84293 Oe0 2.1 I4ol2 23588 Si val 6:45PM OE OLS Asie (esi) 
user2 88467 OO bao = back 1804 5 Sar (eZ et OO) ao Sia ie 
user2 89047 Os 0) aa Oe SOG 780 Bon ie 2ieM 0:00.00 sleep 1 


(Use Ctrl-b and the arrow keys to move to the userl window and type the following) 


ps aux | grep ‘“user2’ 


% 


(userl can no longer view the running processes of the other users.) 


Listing 6. Set the sysctl value for security.mac.seeotheruids.primarygroup_enabled to 1, then try to view the processes of user2 with user]. 
(Note: the while loop in the user2 window should still be echoing “user2") 


(In the root window, type the following) 

7 eVect n SeCiGiny Mec. sesOrneruids. prima rygroupmenanled—| 
ScCUrIEy Maen cceOrnenMids primary IEoupeenaoled= 0. => i: 

# sysctl -a | grep security.mac.see 


& dls 


security.mac.seeotheruids.specificgid: 0 


Secubity Mee .sceornenuvide speciticogid enabled: 0 


SSC biel cy Me. SS owl Sicties jen wie re yeeros eimelelsols 1) 


m 
m 

SecUBIry Macs ceOeneniids ssc priv Leged a) 
m 
m 


security.mac.seeotheruids.enabled: 1 

(Login as user3 in the root window and note that this user cannot see the processes of user2.) 
# su - user3 

ps aux|grep ‘“user2’ 

% 

Sexit 


(Use Ctrl-b and the arrow keys to move to the userl window and type the following) 


%ps aux | grep ‘“user2’ 


IO Oi el OeGr se A196 Ie Sel So: 20EM 0200. Ui si = uiserZ 
user2 7872 OSC a2 ie lee 2324 Sl Oe 0PM 0.0001 =sur (es) 
user2 7876 OOS b= ae PoZ Soot Ooo CS aa e 
wuserZ 36995 ORIOLE wiih! SrS)Ik FIG 2 She 11:16AM 0:00.00 sleep 1 
SpS aux 

USER PID %CPU SMEM VSZ RSs FT STAT STARTED TIME COMMAND 
WOE 7840 OpvO ells  = 4k 296 1648 fe al Mon0SeM 020001 sa — user! 
userl 7841 OO ee a A Gi? 2456 ZS Mon09OPM 0:00.05 =su (esh) 
userl 45557 OU Sed 43S AA Ue 10:43AM 0:00.00 ps aux 
OO Te Wal O70 5 “41296 1648 Sell Mon0SPM 0-00-7010 si — userz 
user2 7872 O02 ae h 2324 3° Mon09PM 0:00.01 =su (esh) 
user2 7876 O20e disc: eiody sul 3 She MomOOr MG) eres cele — 
user2 45556 Of OO 3916 TPO BS 10:43AM 0:00.00 sleep 1 

% 


(userl can now view the running processes of user2 and any processes that belong to the user-reg group.) 
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the module at startup with the use of /boot/loader.conf. additional information provided in the reference section. 
In order to make it easier to monitor different users on the Listing 3 shows the installation steps for tmux using the bi- 
system, tmux will be installed to move between different nary package. Once tmux is installed, type tmux to invoke 
user screens. The basic usage of tmux will be covered with — the terminal multiplexer and type the following: 


Listing 7. Set the sysctl value for security.mac.seeotheruids.specificgid to 1003, which is the gid for the user-reg group then try to view the 
processes of user2 with user]. (Note: the while loop in the user2 window should still be echoing “user2”) 


(In the root window, type the following) 


feeyoce ll ceuiiEy, mac, sceOmNctlico, primar Group enanled—( 
Se Clee y alee. SSO Me oUmcls -joriimlena creole cimelloleole Ik) 

# sysctl security.mac.seeotheruids.specificgid=1003 
security.mac.seeotheruids.specificgid: 0 -> 1003 

fj So yoCe Sccurity. Mac. seccouncriilds so peciice¢ ld emabled=) 
Secuipily Mac ssceORNeuiids Spee meg eG enaoled 0 => 

# 


(Login as user3 in the root window and note that this user cannot see the processes of user2 or any other user.) 


# Su - user3 

SPS aux 

USlailes PID *%CPU MEM VS 4 Roo FF SrA SslARTED TIME COMMAND 
user3 43979 OO 2 ory 2340 es LOT Ave Os VO ROZ = si lest) 
user3 43983 OF ass 1208 Bar 10:18AM 0:00.00 ps aux 
$ps aux | grep ‘“user2’ 

% 


exit 
(Use Ctrl-b and the arrow keys to move to the userl window and type the following) 


ps aux | grep ‘“user2’ 

user2 7872 020° 2a 4612 2324 Si al Mon09PM 0200208 =su (es) 
user2 7876 O20 eo. ao Eo Sap Morag S12) Oreos) OS Iay Tic 
user2 44519 Oe Oy One e Se ke 1S Saeed 10:26AM 0:00.00 sleep 1 


SpS aux 

LOOE DG G.0° 126 18884 1832 0 I+  Mon07PM 0:00.01 tmux: client (/tmp/tmux-0/default) (tmux) 
FOO 1129 OPS ule elec? 2396 es Mon07PM CSO IOS! Srevsilay {lersiial) 
igo, 25.577 OF Oe Oro 1808 eel 10:18AM O00 Ost Weems 
user3 43979 OE 2 enh 2340 2 ise 10:18AM OS R02 sib) (Gers la) 
OIE Ike O20) 2.50) aioe 2296 ES Mon07PM 0200.08 =—esiy (esh) 
“OO 7840 OO so 4256 1648 al Mon09PM 0200.0 tsu = user 
userl 7841 O20 2-2 4G 2456 aes Mon09PM 0200204 =su (esh) 
userl 44709 OO likes Ia {ex0) 2 3 10:29AM 0200200 os) aux 
GLOVE Ih as O20) 2 ao 2304 lai Mon07PM 0200202 =esh (esh) 
OO Teed alk OF OF ees AG 1648 Saal Mon09PM O00 SO sua user 2 
user2 7872 C502 2 aol? 2324 oe al Mon09PM Os 00.01 =su (csh) 
user2 7876 O70 SiGe 4G iReE Sot ee Ment gEenM UU Ole esi eas 
user2 44708 Oe 0 Oy S96 Ws 3 Sap 10:29AM 0:00.00 sleep 1 
OLONE 1163 O20 235 Heol ZoilG 4 Ist Mon09PM 0200207 =csh (esh) 


(userl can now view all processes regardless of uid/gid.) 
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Listing 8. Set the sysctl value for security.mac.seeotheruids.suser_privileged to 1, which restricts the root user from viewing 


(In the root window, type the following) 

W (SyoCl MocCULILy Mac. oo COUNCHUIcS .cUlcor or by Eleged—( 
SecUmilby. Mae soceOrlenUlds -slSeo Oulv i leged. sila=> 10 

# 

f oyoCelpcceit hey Mac. sc OUNCtUICe. sUsor priv leged—( 
Secuimity Mac ssce@ENomiids —sliseosoGnyileged- si a—> 30 

# ps aux|grep ‘“user2’ 

it 

joyce PP secUbi ty tae secOulerl ds olser prt ieleged—| 
SecUublby MaGuoceOElonUlds ssc e soul ylbeged. |= ai 


# ps aux | grep ‘*user2’ 


user2 7872 O20) 2.8 baci? 2324 oa Mon09PM 0200.01 =—su (csh) 

user2 7876 O20 ree 4G eZ Sor Mon09PM POCA col = 1 

user2 46248 OO Ore BoIEG 116 oss 10:54AM 0:00.00 sleep 1 

# 
mean References 
°  Ctri-b ° « FreeBSD Handbook - Mandatory Access Cotnrol: http:// 
* Ctrl-b ° www.freebsd.org/doc/handbook/mac.html 


This will open three window panes in one terminal, as 
seen in Figure 1. 

Using Ctrl-b and the up and down arrow keys allows for 
the movement between window panes. Move to the top 
window, and run ia then move to the middle pane and 
run su - useri then ia. Move to the bottom pane and run 
su - user2 then run ia. The screen should look similar to 
that of Figure 2. 

From this point on, the top, middle and bottom window 
panes will be referred to as the root, user1 and user2 win- 
dows. Listing 4 shows how user1 Is able to view the run- 
ning processes of user2. 

Both user1 and user2 are in the same user-reg group. 
The next step is to turn off the ability to see processes 
from others users and run the same test again. Listing 5 
shows the steps necessary to run this test. 

When loading the MAC module, it doesn't matter if the 
Security. bsd.see other ids is set, as it is overridden by 
secusecurity.mac.seeotheruids.enabled. To allow mem- 
bers of a primary group to see their processes, the security. 
mac.seeotheruids.primarygroup enabled value must be set 
to 1. Listing 6 shows the setting of this value which will allow 
user to view the processes in its primary group. 

Another feature of this module is the ability to exempt a 
group from the restrictions of the uid/gid policy. Listing 7 
shows how to exempt the user-reg group from this policy, 
which will allow user1 and user2 to view all processes on 
the system. 
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« MAC seeotheruids Module: http://www.freebsd.org/doc/ 
handbook/mac-seeotheruids.html 

« Mandatory Access Control: http://en.wikipedia.org/wiki/ 
Mandatory_access_control 

¢ Tmux: http://tmux.sourceforge.net/ 

- TrustedBSD: http:/www.trustedbsd.org/ 


An additional sysctl value can actually restrict the root 
user from viewing the running processes of other users. 
Listing 8 shows the steps necessary to use the feature. 

The examples in this article highlight how to separate 
the viewing of running processes by users in addition to 
the default features included with FreeBSD. The security. 
bsd.see other uids sysctl value can be enabled in the 
default sysctl.conf to provide some user separation as well 
as the default configurations for the mac_seeotheruidsuids 
modules. In later articles, the MAC modules will be com- 
bined to present different layers of security and to help 
with classifying information. 


MICHAEL SHIRK 

Michael Shirk is a BSD zealot who has worked with OpenBSD and 
FreeBSD for over 6 years. He works in the security community 
and supports Open-Source security products that run on BSD op- 
erating systems. 
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OVERVIEW 


his year’s EuroBSDcon and Meet- 
BSD California took place just a few 
weeks apart in two very different loca- 
tions but together demonstrated seamless 
solidarity on the part of the BSD community. 
MeetBSD in Sunnyvale, California was like 
a reunion for many speakers and attendees 
who had recently met in Warsaw, Poland for 
EuroBSDcon. Some familiar European fac- 
es such as Robert Watson and Alexander 
Motin even made appearances only at the 
more distant event, showing once again that 
the geography of BSD and its community is 
“the Internet.” 
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FuroBSDcon with a lesson 


_ about Polish history | 
ames 
— 


Marking its 11th year, EuroBSDcon 2012 chartered new 
territory by being the first of the series to take place in 
“New Europe’, a decision that brought only novelty rather 
than discomfort. Should you choose, you could easily find 


MeetBSD 2012 at Yahoo! 


Starbucks, Subway, McDonald’s and KFC in both cities, 
not to mention overall great food and shopping. The only 
surprise was the thick fog that complicated a few depar- 
tures from Warsaw. 


www.bsdmag.org 


This year’s EuroBSDcon also marked a new milestone 
as being the first client event of the EuroBSDcon Foun- 
dation, a Dutch Stichting that exists to provide legal and 


® financial infrastructure for the migratory conference. This 
_BSD-agnostic body made a distinct impression on the 
event's program by ensuring near-equal representation of 
the leading BSD projects. A mild controversy even sur- 
- rounded the rejection of several OpenBSD proposals due 
to sheer quantity. Who would have thought? 


The FreeBSD Developer Summits that preceded both 
events covered many of the usual topics like the toolchain 
and ports but Alistair Crooks wowed people with a Net- 
flix presentation in Warsaw and | was happy to see the 
BHyVe hypervisor get strong attention in Sunnyvale. Scott 
Long from Netflix continued Alistair’s message at Meet- 
BSD with demonstration hardware to boot: Netflix is cur- 
rently serving over 30% of the traffic on the Internet and is 
moving to an elegant, high-density server that can cache 
its content at ISPs around the world. Remarkably, the so- 
lution is basically a FreeBSD 9.1 web server that distrib- 
utes several terabytes of video files from UFS. The solu- 
tion is very current and very off-the-shelf. The big news for 
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BHyVe is that everyone present agreed that it should be 
merged into the FreeBSD tree as soon as the develop- 
ers see fit. 


fe Dayal cS Gimgnouen Psion el 


_ FreeBSil Beveloper Sexi in Warsaw \ / 


Talk highlights included OpenBSD developer | 
Philip Gunther’s plentiful giving of credit where - 
credit was due on the part of FreeBSD develop- Br hy 
ers, during which Kirk McKusick said, “yep, you 
found a bug we need to fix in FreeBSD.” Martin 
Matuska’s talk about FreeBSD ZFS profiling and 
tuning using tools like /usr/ports/sysutils/zfs- 
stats/ was alSo very good, especially consider- 
ing how little attention these tools have received. « 
John Hixson’s FreeNAS system architecture talk 
provided a nice peek into how FreeNAS works 
under the hood, a topic that has also received 
little attention. Hopefully the videos for all of these 
will be online soon. 

What the EuroBSDcon talks offered in breadth, the 
MeetBSD talks offered in depth. Adrian Chadd took sev- 
eral opportunities to hammer home the point that embed- 
ded FreeBSD has made huge progress in recent months 
and that several near-Tier 1 platforms are available here 
and now. He highlighted the need for a FreeBSD cross- 
compilation environment not unlike NetBSD’s build.sh. 
Being a hot topic, embedded FreeBSD was presented as 
a talk, a full-group discussion and as a dedicated break- 
out session. At first this arrangement seemed like a devia- 
tion from the UnConference format but it turned out to be 
very effective in refining the discussion. 

Most of the MeetBSD presentations are online at hittps:/ 
www.meetbsd.com/conference/talks-and-sessions and 
the EuroBSDcon ones should be up soon. 

Did you miss out? Probably, but perhaps you are not 
aware that various travel grants exist for events like these. 
The organizers of EuroBSDcon, MeetBSD, AsiaBSDCon, 
BSDCan and NYCBSDCon all have travel grants avail- 
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Metflix server design to several 
Seaing Kris Moore 


able for presenters and the FreeBSD Foundation has 
helped dozens of people attend various events over the 
years, including developers from sympathetic projects. 
Google also offered financial support for female comput- 
er scientists to attend EuroBSDcon. Furthermore, | can 
safely say from personal experience that there was a time 
that each and every presenter could have never pictured 
themselves giving a talk at a conference. Events like Eu- 
roBSDon and MeetBSD are the heartbeat of the BSD 
community and | encourage you to find the time to attend 
one, submit a proposal or organize a BSD User 

Group in your area. 


See you at the next BSDCon! 


MICHAEL 


Michael has used BSD Unix systems since 
1991 and is the Editor of the BSD techni- 
cal journal Call For Testing. 
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PgDay.IT 2012 


The sixth edition of the Italian PostgreSQL Day (PgDay) held at 
the Monash University Center in Prato, Tuscany, on November the 
23th has been a success. The Italian community did respond very 
well to the event, and guests from all over the country came to 
discuss, acquire knowledge and share experience about this great 


database. 


Group (ITPUG) is proud of how smooth the sixth 

edition of the Italian PgDay, the national event ded- 
icated to the PostgreSQL database, has been. It was a 
great event, with a lot of attendees from all over the coun- 
try and every detail was simply perfect thanks to the effort 
of all the volunteers and ITPUG members who donated 
their time and effort to the organization of the event. And 
it was not easy: even though ITPUG has been organizing 


Ty he whole staff of the Italian PostgreSQL Users’ 


vi BSD 


PgDay year after year, and it even handled the first Euro- 
pean PgDay back in 2008, scheduling and running such 
an event is not a simple task. Luckily, most of this year’s 
organizers did participate in the organization of previous 
events and therefore shared a common experience on 
tasks that needed to be done in order to make every par- 
ticipant feel comfortable. 

As in the previous edition, the conference was held at 
the Monash University Center in Prato, Tuscany, in a great 


fe J ¢ | j : 
Group picture of the Italian PgDay 2012 
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building where two rooms, the Grollo and Veneziana, 
were prepared with appropriate devices for each speech. 

It is worth noting that, even if the name “PgDay” sounds 
like a “one-day” event, this is not the truth about the 2012 
PgDay (and previous events too). In fact, the community 
gathered the evening before in a local pub to enjoy a “Pg- 
Beer” offered by one of the conference sponsors. It was 
the perfect place to meet other professionals and passion- 
ates and share some experiences and laugh at scary and 
strange stories (all based on database tales, of course!). 
The evening continued in a local restaurant, where the 
participants enjoyed a delicious Fiorentina steak, a very 
famous kind of meal in Tuscany. Again, a perfect way to sit 
down and talk to other people, all peers, and share experi- 
ences, opinions, tips and tricks and so on. 

The day after, of course, the conference took place with 
the opening session from the ITPUG’s president Gabriele 
Bartolini. The attendees then listened in a kind of religious 
respect to the keynote talk by Simon Riggs and Andres 
Freund that explained the ongoing work for multi-master 
replication, a feature that will take several releases to get 
fully implemented in PostgreSQL and that will make an- 
other giant step for this database in the race to be the 
leader on the SQL market. 

A short coffee break served along with local pastry, and 
then two parallel sessions began. The Veneziana room 
was dedicated to tutorial sessions, with two introductory 
sessions from yours truly and two sessions on the de- 
velopment of stored procedures. In the meantime, in the 
Grollo room it was time to introduce new features coming 
for free with the current 9.2 release, migration from Oracle 
to PostgreSQL and techniques to monitor and keep Post- 
greSQL instances healthy. 

Of course time flies when you have a program full of 
such talks, and a lot of attendees were literally jumping 
from one room to the other in order to get even a single 
bit of information...and then it was time for lunch. The buf- 
fet lunch was another demonstration of the Tuscany su- 


PgDay.IT 2012 by numbers 


At the 2012 Italian PgDay there were 95 attendees, including 
8 regular speakers, and a few lightning talk speakers. Seven- 
ty-eight percent of the attendees came from northern Ita- 
ly, including Emilia Romagna at its edge, while the rest came 
from central and southern Italy. Fifteen regular talks were giv- 
en during the day, including the technical keynote. The con- 
ference was organized with the help of two gold sponsors and 
one bronze, and the patronage of the local city and a univer- 
sity Open Source laboratory. The on-site dedicated staff was 
made up of 7 volunteers, and other ITPUG members joined 
the staff on demand. 
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On the Web 
Italian PostgreSQL Users’ Group (ITPUG): http://www. itpug.org 


PgDay.IT 2012: http://2012.pgday.it 
PostgreSQL: http:/www.postgresql.org 


periority when taking down to meal. And it was really nice 
to walk around and see a lot of people not only enjoy- 
ing great food, but again talking and sharing experiences, 
providing each other suggestions to solve some specific 
problems, and so on. The atmosphere was really relaxed 
and there was time to joke around and even take a group 
picture with all the members of the staff and all the at- 
tendees. 

The afternoon was again filled with two parallel ses- 
sions: the Veneziana room focused on database devel- 
opment with talks about database unit testing, log analy- 
sis and database design for high volumes of data. In the 
other room, experiences and case studies related to Java 
EE and the adoption of PostgreSQL for High Availability 
solutions in Italian health-care (two talks) captured the at- 
tention. 

As a tradition so far, approaching the end of the event 
there was a unique session of lightning talks, talks that 
can be no longer than 5 minutes and that can be on al- 
most any subject, idea, claim, consideration, experience, 
and so on related to the PostgreSQL (or the database in 
general) world. 

Two lucky attendees won a signed copy of the latest 
PostgreSQL books, donated by one of the event spon- 
sors. The conference ended on time, with the closing ses- 
sion and a recap of the day by the author. 

And as years before, while the official event was at the 
end, the community one was not. In fact, no more than 
20 minutes later, the PostgreSQL addicted were populat- 
ing the nearby pub drinking another great PgBeer offered 
again from a conference sponsor. 

I'd like to thank all the organizers for their great and pro- 
fessional activity, as well as all the sponsors, and all the 
speakers for their quality contributions, but most notably 
every single attendee for trusting in ITPUG and the PgDay 
and for letting PostgreSQL be such a great product. 

See you at PgDay.IT 2013! 


LUCA FERRARI 

Luca Ferrari lives in Italy with his wife and son. He is an Ad- 
junct Professor at Nipissing University, Canada, a co-founder 
and the vice-president of the Italian PostgreSQL Users’ Group 
(ITPUG). He simply loves the Open Source culture and refus- 
es to log-in to non-Unix systems. He can be reached on line at 
http://fluca1978.blogspot.com. 
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